Backpressure control using a stop signal for a multi-threaded, self-scheduling reconfigurable computing fabric

ABSTRACT

Representative apparatus, method, and system embodiments are disclosed for configurable computing. A representative system includes an interconnection network; a processor; and a plurality of configurable circuit clusters. Each configurable circuit cluster includes a plurality of configurable circuits arranged in an array; a synchronous network coupled to each configurable circuit of the array; and an asynchronous packet network coupled to each configurable circuit of the array. A representative configurable circuit includes a configurable computation circuit and a configuration memory having a first, instruction memory storing a plurality of data path configuration instructions to configure a data path of the configurable computation circuit; and a second, instruction and instruction index memory storing a plurality of spoke instructions and data path configuration instruction indices for selection of a master synchronous input, a current data path configuration instruction, and a next data path configuration instruction for a next configurable computation circuit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit of andpriority to U.S. patent application Ser. No. 16/371,045, filed Mar. 31,2019, inventor Tony M. Brewer, titled “Backpressure Control Using a StopSignal for a Multi-Threaded, Self-Scheduling Reconfigurable ComputingFabric”, which is a nonprovisional of and claims the benefit of andpriority to U.S. Provisional Patent Application No. 62/651,131, filedMar. 31, 2018, inventor Tony M. Brewer, titled “Backpressure ControlUsing a Stop Signal for a Multi-Threaded, Self-Scheduling ReconfigurableComputing Fabric”, which are commonly assigned herewith, and all ofwhich are hereby incorporated herein by reference in their entiretieswith the same full force and effect as if set forth in their entiretiesherein.

FIELD OF THE INVENTION

The present invention, in general, relates to configurable computingcircuitry, and more particularly, relates to a heterogeneous computingsystem which includes configurable computing circuitry with an embeddedinterconnection network, dynamic reconfiguration, and dynamic controlover energy or power consumption.

BACKGROUND OF THE INVENTION

Many existing computing systems have reached significant limits forcomputation processing capabilities, both in terms of speed ofcomputation, energy (or power) consumption, and associated heatdissipation. For example, existing computing solutions have becomeincreasingly inadequate as the need for advanced computing technologiesgrows, such as to accommodate artificial intelligence and othersignificant computing applications.

Accordingly, there is an ongoing need for a computing architecturecapable of providing high performance and energy efficient solutions forcompute-intensive kernels, such as for computation of Fast FourierTransforms (FFTs) and finite impulse response (FIR) filters used insensing, communication, and analytic applications, such as syntheticaperture radar, 5G base stations, and graph analytic applications suchas graph clustering using spectral techniques, machine learning, 5Gnetworking algorithms, and large stencil codes, for example and withoutlimitation.

There is also an ongoing need for a configurable computing architecturecapable of being configured for any of these various applications, butmost importantly, also capable of dynamic self-configuration andself-reconfiguration.

SUMMARY OF THE INVENTION

As discussed in greater detail below, the representative apparatus,system and method provide for a computing architecture capable ofproviding high performance and energy efficient solutions forcompute-intensive kernels, such as for computation of Fast FourierTransforms (FFTs) and finite impulse response (FIR) filters used insensing, communication, and analytic applications, such as syntheticaperture radar, 5G base stations, and graph analytic applications suchas graph clustering using spectral techniques, machine learning, 5Gnetworking algorithms, and large stencil codes, for example and withoutlimitation.

Significantly, the various representative embodiments provide amulti-threaded, coarse-grained configurable computing architecturecapable of being configured for any of these various applications, butmost importantly, also capable of self-scheduling, dynamicself-configuration and self-reconfiguration, conditional branching,backpressure control for asynchronous signaling, ordered threadexecution and loop thread execution (including with data dependencies),automatically starting thread execution upon completion of datadependencies and/or ordering, providing loop access to privatevariables, providing rapid execution of loop threads using a reenterqueue, and using various thread identifiers for advanced loop execution,including nested loops.

In a representative embodiment, a configurable circuit may comprise: aconfigurable computation circuit; a plurality of synchronous networkinputs coupled to the configurable computation circuit; a plurality ofsynchronous network outputs coupled to the configurable computationcircuit; and a configuration memory coupled to the configurablecomputation circuit, to the control circuitry, to the synchronousnetwork inputs, and to the synchronous network outputs, with theconfiguration memory comprising: a first, instruction memory storing aplurality of data path configuration instructions to configure a datapath of the configurable computation circuit; and a second, instructionand instruction index memory storing a plurality of spoke instructionsand data path configuration instruction indices for selection of amaster synchronous input of the synchronous network inputs.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; and a configuration memorycoupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs, the configuration memory comprising: a first,instruction memory storing a plurality of data path configurationinstructions to configure a data path of the configurable computationcircuit; and a second, instruction and instruction index memory storinga plurality of spoke instructions and data path configurationinstruction indices for selection of a current data path configurationinstruction for the configurable computation circuit.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; and a configuration memorycoupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs, the configuration memory comprising: a first,instruction memory storing a plurality of data path configurationinstructions to configure a data path of the configurable computationcircuit; and a second, instruction and instruction index memory storinga plurality of spoke instructions and data path configurationinstruction indices for selection of a next data path configurationinstruction for a next configurable computation circuit.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a control circuit coupledto the configurable computation circuit; a first memory circuit coupledto the configurable computation circuit; a plurality of synchronousnetwork inputs coupled to the configurable computation circuit; aplurality of synchronous network outputs coupled to the configurablecomputation circuit; and a second, configuration memory circuit coupledto the configurable computation circuit, to the control circuitry, tothe synchronous network inputs, and to the synchronous network outputs,the configuration memory circuit comprising: a first, instruction memorystoring a plurality of data path configuration instructions to configurea data path of the configurable computation circuit; and a second,instruction and instruction index memory storing a plurality of spokeinstructions and data path configuration instruction indices forselection of a master synchronous input of the synchronous networkinputs.

In yet another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a first memory circuitcoupled to the configurable computation circuit; a plurality ofsynchronous network inputs coupled to the configurable computationcircuit; a plurality of synchronous network outputs coupled to theconfigurable computation circuit; and a second, configuration memorycircuit coupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs; and a control circuit coupled to the configurablecomputation circuit, the control circuit comprising: a memory controlcircuit; a thread control circuit; and a plurality of control registers.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a configuration memorycoupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs, the configuration memory comprising: a first,instruction memory storing a plurality of data path configurationinstructions to configure a data path of the configurable computationcircuit; and a second, instruction and instruction index memory storinga plurality of spoke instructions and data path configurationinstruction indices for selection of a next data path instruction ornext data path instruction index for a next configurable computationcircuit; and a conditional logic circuit coupled to the configurablecomputing circuit, wherein depending upon an output from theconfigurable computing circuit, the conditional logic circuit is adaptedto provide conditional branching by modifying the next data pathinstruction or next data path instruction index provided on a selectedoutput of the plurality of synchronous network outputs.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a control circuit coupledto the configurable computation circuit; a first memory circuit coupledto the configurable computation circuit; a plurality of synchronousnetwork inputs coupled to the configurable computation circuit; aplurality of synchronous network outputs coupled to the configurablecomputation circuit; an asynchronous network input queue coupled to anasynchronous packet network and to the first memory circuit; anasynchronous network output queue; and a flow control circuit coupled tothe asynchronous network output queue, the flow control circuit adaptedto generate a stop signal when a predetermined threshold has beenreached in the asynchronous network output queue.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a first memory circuitcoupled to the configurable computation circuit; a plurality ofsynchronous network inputs coupled to the configurable computationcircuit; a plurality of synchronous network outputs coupled to theconfigurable computation circuit; and a second, configuration memorycircuit coupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs; and a control circuit coupled to the configurablecomputation circuit, the control circuit comprising: a memory controlcircuit; a thread control circuit; and a plurality of control registers,wherein the plurality of control registers store a loop table having aplurality of thread identifiers and, for each thread identifier, a nextthread identifier for execution following execution of a current threadto provide ordered thread execution.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a first memory circuitcoupled to the configurable computation circuit; a plurality ofsynchronous network inputs coupled to the configurable computationcircuit; a plurality of synchronous network outputs coupled to theconfigurable computation circuit; and a second, configuration memorycircuit coupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs; and a control circuit coupled to the configurablecomputation circuit, the control circuit comprising: a memory controlcircuit; a plurality of control registers, wherein the plurality ofcontrol registers store a completion table having a first, datacompletion count; and a thread control circuit adapted to queue a threadfor execution when, for its thread identifier, its completion count hasdecremented to zero.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a first memory circuitcoupled to the configurable computation circuit; a plurality ofsynchronous network inputs and outputs coupled to the configurablecomputation circuit; an asynchronous network input queue and anasynchronous network output queue; a second, configuration memorycircuit coupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs, the second, configuration memory comprising: a first,instruction memory storing a plurality of data path configurationinstructions to configure a data path of the configurable computationcircuit; and a second, instruction and instruction index memory storing:a plurality of spoke instructions and data path configurationinstruction indices for selection of a master synchronous input of thesynchronous network inputs, for selection of a current data pathconfiguration instruction for the configurable computation circuit, andfor selection of a next data path instruction or next data pathinstruction index for a next configurable computation circuit; and theconfigurable circuit further comprising a control circuit coupled to theconfigurable computation circuit, the control circuit comprising: amemory control circuit; a plurality of control registers, wherein theplurality of control registers store a completion table having a first,data completion count; and a thread control circuit adapted to queue athread for execution when, for its thread identifier, its completioncount has decremented to zero.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a first memory circuitcoupled to the configurable computation circuit; a plurality ofsynchronous network inputs coupled to the configurable computationcircuit; a plurality of synchronous network outputs coupled to theconfigurable computation circuit; and a second, configuration memorycircuit coupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs; and a control circuit coupled to the configurablecomputation circuit, the control circuit comprising: a memory controlcircuit; a plurality of control registers, wherein the plurality ofcontrol registers store a completion table having a first, datacompletion count; and a thread control circuit adapted to queue a threadfor execution when, for its thread identifier, its completion count hasdecremented to zero and its thread identifier is the next thread.

In yet another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a first memory circuitcoupled to the configurable computation circuit; a plurality ofsynchronous network inputs coupled to the configurable computationcircuit; a plurality of synchronous network outputs coupled to theconfigurable computation circuit; and a second, configuration memorycircuit coupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs; and the configurable circuit further comprising acontrol circuit coupled to the configurable computation circuit, thecontrol circuit comprising: a memory control circuit; a thread controlcircuit; and a plurality of control registers storing a completion tablehaving a plurality of types of thread identifiers, with each type ofthread identifier indicating a loop level for loop and nested loopexecution, and wherein the plurality of control registers further storea top of thread identifiers stack to allow each type of threadidentifier access to private variables for a selected loop.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a first memory circuitcoupled to the configurable computation circuit; a plurality ofsynchronous network inputs coupled to the configurable computationcircuit; a plurality of synchronous network outputs coupled to theconfigurable computation circuit; and a second, configuration memorycircuit coupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs; and a control circuit coupled to the configurablecomputation circuit, the control circuit comprising: a memory controlcircuit; a plurality of control registers; and a thread control circuitcomprising: a continuation queue storing one or more thread identifiersfor computation threads having completion counts allowing execution butdo not yet have an assigned thread identifier; and a reenter queuestoring one or more thread identifiers for computation threads havingcompletion counts allowing execution and having an assigned threadidentifier to provide for execution of the threads in the reenter queueupon a designated spoke count.

In another representative embodiment, a configurable circuit maycomprise: a configurable computation circuit; a first memory circuitcoupled to the configurable computation circuit; a plurality ofsynchronous network inputs coupled to the configurable computationcircuit; a plurality of synchronous network outputs coupled to theconfigurable computation circuit; and a second, configuration memorycircuit coupled to the configurable computation circuit, to the controlcircuitry, to the synchronous network inputs, and to the synchronousnetwork outputs; and a control circuit coupled to the configurablecomputation circuit, the control circuit comprising: a memory controlcircuit; a plurality of control registers storing a thread identifierpool and a completion table having a loop count of an active number ofloop threads; and a thread control circuit, wherein in response toreceipt of an asynchronous fabric message returning a thread identifierto the thread identifier pool, the control circuit decrements the loopcount and, when the loop count reaches zero, transmits an asynchronousfabric completion message.

In a representative embodiment, a system is disclosed, which maycomprise: an asynchronous packet network; a synchronous network; and aplurality of configurable circuits arranged in an array, eachconfigurable circuit of the plurality of configurable circuits coupledto both the synchronous network and to the asynchronous packet network,the plurality of configurable circuits adapted to perform a plurality ofcomputations using the synchronous network to form a plurality ofsynchronous domains, and the plurality of configurable circuits furtheradapted to generate and transmit a plurality of control messages overthe asynchronous packet network, the plurality of control messagescomprising one or more completion messages and continue messages.

In another representative embodiment, a system may comprise: a pluralityof configurable circuits arranged in an array; a synchronous networkcoupled to each configurable circuit of the plurality of configurablecircuits of the array; and an asynchronous packet network coupled toeach configurable circuit of the plurality of configurable circuits ofthe array.

In another representative embodiment, a system may comprise: aninterconnection network; a processor coupled to the interconnectionnetwork; and a plurality of configurable circuit clusters coupled to theinterconnection network.

In a representative embodiment, a system may comprise: aninterconnection network; a processor coupled to the interconnectionnetwork; a host interface coupled to the interconnection network; and aplurality of configurable circuit clusters coupled to theinterconnection network, each configurable circuit cluster of theplurality of configurable circuit clusters comprising: a plurality ofconfigurable circuits arranged in an array; a synchronous networkcoupled to each configurable circuit of the plurality of configurablecircuits of the array; an asynchronous packet network coupled to eachconfigurable circuit of the plurality of configurable circuits of thearray; a memory interface coupled to the asynchronous packet network andto the interconnection network; and a dispatch interface coupled to theasynchronous packet network and to the interconnection network.

In another representative embodiment, a system may comprise: ahierarchical interconnection network comprising a first plurality ofcrossbar switches having a Folded Clos configuration and a plurality ofdirect, mesh connections at interfaces with endpoints; a processorcoupled to the interconnection network; a host interface coupled to theinterconnection network; and a plurality of configurable circuitclusters coupled to the interconnection network, each configurablecircuit cluster of the plurality of configurable circuit clusterscomprising: a plurality of configurable circuits arranged in an array; asynchronous network coupled to each configurable circuit of theplurality of configurable circuits of the array and providing aplurality of direct connections between adjacent configurable circuitsof the array; an asynchronous packet network comprising a secondplurality of crossbar switches, each crossbar switch coupled to at leastone configurable circuit of the plurality of configurable circuits ofthe array and to another crossbar switch of the second plurality ofcrossbar switches; a memory interface coupled to the asynchronous packetnetwork and to the interconnection network; and a dispatch interfacecoupled to the asynchronous packet network and to the interconnectionnetwork.

In another representative embodiment, a system may comprise: aninterconnection network; a processor coupled to the interconnectionnetwork; a host interface coupled to the interconnection network; and aplurality of configurable circuit clusters coupled to theinterconnection network, each configurable circuit cluster of theplurality of configurable circuit clusters comprising: a synchronousnetwork; an asynchronous packet network; a memory interface coupled tothe asynchronous packet network and to the interconnection network; adispatch interface coupled to the asynchronous packet network and to theinterconnection network; and a plurality of configurable circuitsarranged in an array, each configurable circuit comprising: aconfigurable computation circuit; a control circuit coupled to theconfigurable computation circuit, the control circuit comprising: amemory control circuit; a thread control circuit; and a plurality ofcontrol registers; a first memory circuit coupled to the configurablecomputation circuit; a plurality of synchronous network inputs andoutputs coupled to the configurable computation circuit and to thesynchronous network; an asynchronous network input queue and anasynchronous network output queue coupled to the asynchronous packetnetwork; a second, configuration memory circuit coupled to theconfigurable computation circuit, to the control circuitry, to thesynchronous network inputs, and to the synchronous network outputs, theconfiguration memory circuit comprising: a first, instruction memorystoring a plurality of data path configuration instructions to configurea data path of the configurable computation circuit; and a second,instruction and instruction index memory storing a plurality of spokeinstructions and data path configuration instruction indices forselection of a master synchronous input of the synchronous networkinputs.

In any of the various representative embodiments, the second,instruction and instruction index memory may further store a pluralityof spoke instructions and data path configuration instruction indicesfor selection of a current data path configuration instruction for theconfigurable computation circuit.

In any of the various representative embodiments, the second,instruction and instruction index memory may further store a pluralityof spoke instructions and data path configuration instruction indicesfor selection of a next data path configuration instruction for a nextconfigurable computation circuit.

In any of the various representative embodiments, the second,instruction and instruction index memory may further store a pluralityof spoke instructions and data path configuration instruction indicesfor selection of a synchronous network output of the plurality ofsynchronous network outputs.

In any of the various representative embodiments, the configurablecircuit or system may further comprise: a configuration memorymultiplexer coupled to the first, instruction memory and to the second,instruction and instruction index memory.

In any of the various representative embodiments, when a selection inputof the configuration memory multiplexer has a first setting, the currentdata path configuration instruction may be selected using an instructionindex from the second, instruction and instruction index memory.

In any of the various representative embodiments, when the selectioninput of the configuration memory multiplexer has a second settingdifferent from the first setting, the current data path configurationinstruction may be selected using an instruction index from the mastersynchronous input.

In any of the various representative embodiments, the second,instruction and instruction index memory may further store a pluralityof spoke instructions and data path configuration instruction indicesfor configuration of portions of the configurable circuit independentlyfrom the current data path instruction.

In any of the various representative embodiments, a selected spokeinstruction and data path configuration instruction index of theplurality of spoke instructions and data path configuration instructionindices may be selected according to a modulo spoke count.

In any of the various representative embodiments, the configurablecircuit or system may further comprise: a conditional logic circuitcoupled to the configurable computing circuit.

In any of the various representative embodiments, depending upon anoutput from the configurable computing circuit, the conditional logiccircuit may be adapted to modify the next data path instruction indexprovided on a selected output of the plurality of synchronous networkoutputs.

In any of the various representative embodiments, depending upon anoutput from the configurable computing circuit, the conditional logiccircuit may be adapted to provide conditional branching by modifying thenext data path instruction or next data path instruction index providedon a selected output of the plurality of synchronous network outputs.

In any of the various representative embodiments, the conditional logiccircuit, when enabled, may be adapted to provide conditional branchingby ORing the least significant bit of the next data path instructionwith the output from the configurable computing circuit to designate thenext data path instruction or data path instruction index.

In any of the various representative embodiments, the conditional logiccircuit, when enabled, may be adapted to provide conditional branchingby ORing the least significant bit of the next data path instructionindex with the output from the configurable computing circuit todesignate the next data path instruction index.

In any of the various representative embodiments, the plurality ofsynchronous network inputs may comprise: a plurality of input registerscoupled to a plurality of communication lines of a synchronous network;and an input multiplexer coupled to the plurality of input registers andto the second, instruction and instruction index memory for selection ofthe master synchronous input.

In any of the various representative embodiments, the plurality ofsynchronous network outputs may comprise: a plurality of outputregisters coupled to a plurality of communication lines of thesynchronous network; and an output multiplexer coupled to theconfigurable computing circuit for selection of an output from theconfigurable computing circuit.

In any of the various representative embodiments, the configurablecircuit or system may further comprise: an asynchronous network inputqueue coupled to an asynchronous packet network and to the memorycircuit; and an asynchronous network output queue coupled to the outputmultiplexer.

In any of the various representative embodiments, the configurablecircuit or system may further comprise: an asynchronous fabric statemachine coupled to the asynchronous network input queue and to theasynchronous network output queue, the asynchronous fabric state machineadapted to decode an input data packet received from the asynchronouspacket network and to assemble an output data packet for transmission onthe asynchronous packet network.

In any of the various representative embodiments, the asynchronouspacket network may comprise a plurality of crossbar switches, eachcrossbar switch coupled to a plurality of configurable circuits and toat least one other crossbar switch.

In any of the various representative embodiments, the configurablecircuit or system may further comprise: an array of a plurality ofconfigurable circuits, wherein: each configurable circuit is coupledthrough the plurality of synchronous network inputs and the plurality ofsynchronous network outputs to the synchronous network; and eachconfigurable circuit is coupled through the asynchronous network inputand the asynchronous network output to the asynchronous packet network.

In any of the various representative embodiments, the synchronousnetwork may comprise a plurality of direct point-to-point connectionscoupling adjacent configurable circuits of the array of the plurality ofconfigurable circuits.

In any of the various representative embodiments, each configurablecircuit may comprise: a direct, pass through connection between theplurality of input registers and the plurality of output registers. Inany of the various representative embodiments, the direct, pass throughconnection may provide a direct, point-to-point connection for datatransmission from a second configurable circuit received on thesynchronous network to a third configurable circuit transmitted on thesynchronous network.

In any of the various representative embodiments, the configurablecomputation circuit may comprise an arithmetic, logical and bitoperation circuit adapted to perform at least one integer operationselected from the group consisting of: signed and unsigned addition,absolute value, negate, logical NOT, add and negate, subtraction A-B,reverse subtraction B-A, signed and unsigned greater than, signed andunsigned greater than or equal to, signed and unsigned less than, signedand unsigned less than or equal to, comparison of equal or not equal to,logical AND operation, logical OR operation, logical XOR operation,logical NAND operation, logical NOR operation, logical NOT XORoperation, logical AND NOT operation, logical OR NOT operation, and aninterconversion between integer and floating point.

In any of the various representative embodiments, the configurablecomputation circuit may comprise an arithmetic, logical and bitoperation circuit adapted to perform at least one floating pointoperation selected from the group consisting of: signed and unsignedaddition, absolute value, negate, logical NOT, add and negate,subtraction A-B, reverse subtraction B-A, signed and unsigned greaterthan, signed and unsigned greater than or equal to, signed and unsignedless than, signed and unsigned less than or equal to, comparison ofequal or not equal to, logical AND operation, logical OR operation,logical XOR operation, logical NAND operation, logical NOR operation,logical NOT XOR operation, logical AND NOT operation, logical OR NOToperation, an interconversion between integer and floating point, andcombinations thereof.

In any of the various representative embodiments, the configurablecomputation circuit may comprise a multiply and shift operation circuitadapted to perform at least one integer operation selected from thegroup consisting of: multiply, shift, pass an input, signed and unsignedmultiply, signed and unsigned shift right, signed and unsigned shiftleft, bit order reversal, a permutation, an interconversion betweeninteger and floating point, and combinations thereof.

In any of the various representative embodiments, the configurablecomputation circuit may comprise a multiply and shift operation circuitadapted to perform at least floating point operation selected from thegroup consisting of: multiply, shift, pass an input, signed and unsignedmultiply, signed and unsigned shift right, signed and unsigned shiftleft, bit order reversal, a permutation, an interconversion betweeninteger and floating point, and combinations thereof.

In any of the various representative embodiments, the array of theplurality of configurable circuits may be further coupled to a firstinterconnection network. In any of the various representativeembodiments, the array of the plurality of configurable circuits mayfurther comprise: a third, system memory interface circuit; and adispatch interface circuit. In any of the various representativeembodiments, the dispatch interface circuit may be adapted to receive awork descriptor packet over the first interconnection network, and inresponse to the work descriptor packet, to generate one or more data andcontrol packets to the plurality of configurable circuits to configurethe plurality of configurable circuits for execution of a selectedcomputation.

In any of the various representative embodiments, the configurablecircuit or system may further comprise: a flow control circuit coupledto the asynchronous network output queue, the flow control circuitadapted to generate a stop signal when a predetermined threshold hasbeen reached in the asynchronous network output queue. In any of thevarious representative embodiments, in response to the stop signal, eachasynchronous network output queue stops outputting data packets on theasynchronous packet network. In any of the various representativeembodiments, in response to the stop signal, each configurablecomputation circuit stops executing upon completion of its currentinstruction.

In any of the various representative embodiments, a first plurality ofconfigurable circuits of the array of a plurality of configurablecircuits may be coupled in a first predetermined sequence through thesynchronous network to form a first synchronous domain; and wherein asecond plurality of configurable circuits of the array of a plurality ofconfigurable circuits are coupled in a second predetermined sequencethrough the synchronous network to form a second synchronous domain. Inany of the various representative embodiments, the first synchronousdomain may be adapted to generate a continuation message to the secondsynchronous domain transmitted through the asynchronous packet network.In any of the various representative embodiments, the second synchronousdomain may be adapted to generate a completion message to the firstsynchronous domain transmitted through the asynchronous packet network.

In any of the various representative embodiments, the plurality ofcontrol registers may store a completion table having a first, datacompletion count. In any of the various representative embodiments, theplurality of control registers further store the completion table havinga second, iteration count. In any of the various representativeembodiments, the plurality of control registers may further store a looptable having a plurality of thread identifiers and, for each threadidentifier, a next thread identifier for execution following executionof a current thread. In any of the various representative embodiments,the plurality of control registers may further store, in the loop table,an identification of a first iteration and an identification of a lastiteration.

In any of the various representative embodiments, the control circuitmay be adapted to queue a thread for execution when, for its threadidentifier, its completion count has decremented to zero and its threadidentifier is the next thread.

In any of the various representative embodiments, the control circuitmay be adapted to queue a thread for execution when, for its threadidentifier, its completion count indicates completion of any datadependencies. In any of the various representative embodiments, thecompletion count may indicate a predetermined number of completionmessages to be received, per selected thread of a plurality of threads,prior to execution of the selected thread.

In any of the various representative embodiments, the plurality ofcontrol registers may further store a completion table having aplurality of types of thread identifiers, with each type of threadidentifier indicating a loop level for loop and nested loop execution.

In any of the various representative embodiments, the plurality ofcontrol registers may further store a completion table having a loopcount of an active number of loop threads, and wherein in response toreceipt of an asynchronous fabric message returning a thread identifierto a thread identifier pool, the control circuit decrements the loopcount and, when the loop count reaches zero, transmits an asynchronousfabric completion message. In any of the various representativeembodiments, the plurality of control registers may further store a topof thread identifiers stack to allow each type of thread identifieraccess to private variables for a selected loop.

In any of the various representative embodiments, the control circuitmay further comprise: a continuation queue; and a reenter queue. In anyof the various representative embodiments, the continuation queue storesone or more thread identifiers for computation threads having completioncounts allowing execution but do not yet have an assigned threadidentifier. In any of the various representative embodiments, thereenter queue may store one or more thread identifiers for computationthreads having completion counts allowing execution and having anassigned thread identifier. In any of the various representativeembodiments, any thread having a thread identifier in the reenter queuemay be executed prior to execution of any thread having a threadidentifier in the continuation queue.

In any of the various representative embodiments, the control circuitmay further comprise: a priority queue, wherein any thread having athread identifier in the priority queue may be executed prior toexecution of any thread having a thread identifier in the continuationqueue or in the reenter queue.

In any of the various representative embodiments, the control circuitmay further comprise: a run queue, wherein any thread having a threadidentifier in the run queue may be executed upon an occurrence of aspoke count for the thread identifier.

In any of the various representative embodiments, the second,configuration memory circuit may comprise: a first, instruction memorystoring a plurality of data path configuration instructions to configurea data path of the configurable computation circuit; and a second,instruction and instruction index memory storing a plurality of spokeinstructions and data path configuration instruction indices forselection of a master synchronous input of the synchronous networkinputs.

In any of the various representative embodiments, the control circuitmay be adapted to self-schedule a computation thread for execution.

In any of the various representative embodiments, the conditional logiccircuit may be adapted to branch to a different, second next instructionfor execution by a next configurable circuit.

In any of the various representative embodiments, the control circuitmay be adapted to order computation threads for execution. In any of thevarious representative embodiments, the control circuit may be adaptedto order loop computation threads for execution. In any of the variousrepresentative embodiments, the control circuit may be adapted tocommence execution of computation threads in response to one or morecompletion signals from data dependencies.

Various method embodiments of configuring a configurable circuit arealso disclosed. A representative method embodiment may comprise: using afirst, instruction memory, providing a plurality of data pathconfiguration instructions to configure a data path of the configurablecomputation circuit; and using a second, instruction and instructionindex memory, providing a plurality of spoke instructions and data pathconfiguration instruction indices for selection of a master synchronousinput of a plurality of synchronous network inputs.

In any of the various representative embodiments, a method ofconfiguring a configurable circuit may comprise: using a first,instruction memory, providing a plurality of data path configurationinstructions to configure a data path of the configurable computationcircuit; and using a second, instruction and instruction index memory,providing a plurality of spoke instructions and data path configurationinstruction indices for selection of a current data path configurationinstruction for the configurable computation circuit.

In any of the various representative embodiments, a method ofconfiguring a configurable circuit may comprise: using a first,instruction memory, providing a plurality of data path configurationinstructions to configure a data path of the configurable computationcircuit; and using a second, instruction and instruction index memory,providing a plurality of spoke instructions and data path configurationinstruction indices for selection of a next data path configurationinstruction for a next configurable computation circuit.

A method of controlling thread execution of a multi-threadedconfigurable circuit is also disclosed, with the configurable circuithaving a configurable computation circuit. A representative methodembodiment may comprise: using a conditional logic circuit, dependingupon an output from the configurable computing circuit, providingconditional branching by modifying the next data path instruction ornext data path instruction index provided to a next configurablecircuit.

Another representative method embodiment of controlling thread executionof a multi-threaded configurable circuit may comprise: using a flowcontrol circuit, generating a stop signal when a predetermined thresholdhas been reached in an asynchronous network output queue.

Another representative method embodiment of controlling thread executionof a multi-threaded configurable circuit may comprise: using a pluralityof control registers, storing a loop table having a plurality of threadidentifiers and, for each thread identifier, a next thread identifierfor execution following execution of a current thread to provide orderedthread execution.

Another representative method embodiment of controlling thread executionof a multi-threaded configurable circuit may comprise: using a pluralityof control registers, storing a completion table having a first, datacompletion count; and using a thread control circuit, queueing a threadfor execution when, for its thread identifier, its completion count hasdecremented to zero.

A method of configuring and controlling thread execution of amulti-threaded configurable circuit having a configurable computationcircuit is disclosed, with a representative method embodimentcomprising: using a first, instruction memory, providing a plurality ofdata path using configuration instructions to configure a data path ofthe configurable computation circuit; using a second, instruction andinstruction index memory, providing a plurality of spoke instructionsand data path configuration instruction indices for selection of amaster synchronous input of a plurality of synchronous network inputs,for selection of a current data path configuration instruction for theconfigurable computation circuit, and for selection of a next data pathinstruction or next data path instruction index for a next configurablecomputation circuit; using a plurality of control registers, providing acompletion table having a first, data completion count; and using athread control circuit, queueing a thread for execution when, for itsthread identifier, its completion count has decremented to zero.

Another method of configuring and controlling thread execution of amulti-threaded configurable circuit may comprise: using a first,instruction memory, providing a plurality of data path usingconfiguration instructions to configure a data path of the configurablecomputation circuit; using a second, instruction and instruction indexmemory, providing a plurality of spoke instructions and data pathconfiguration instruction indices for selection of a master synchronousinput of a plurality of synchronous network inputs, for selection of acurrent data path configuration instruction for the configurablecomputation circuit, and for selection of a next data path instructionor next data path instruction index for a next configurable computationcircuit; using a plurality of control registers, providing a completiontable having a first, data completion count; and using a thread controlcircuit, queueing a thread for execution when, for its threadidentifier, its completion count has decremented to zero and its threadidentifier is the next thread.

Another method of controlling thread execution of a multi-threadedconfigurable circuit may comprise: using a plurality of controlregisters, storing a completion table having a plurality of types ofthread identifiers, with each type of thread identifier indicating aloop level for loop and nested loop execution, and wherein the pluralityof control registers further store a top of thread identifiers stack;and allowing each type of thread identifier access to private variablesfor a selected loop.

Another method of controlling thread execution of a multi-threadedconfigurable circuit may comprise: using a plurality of controlregisters, storing a completion table having a data completion count;using a thread control circuit, providing a continuation queue storingone or more thread identifiers for computation threads having completioncounts allowing execution but do not yet have an assigned threadidentifier; and using a thread control circuit, providing a reenterqueue storing one or more thread identifiers for computation threadshaving completion counts allowing execution and having an assignedthread identifier to provide for execution of the threads in the reenterqueue upon a designated spoke count.

Another method of controlling thread execution of a multi-threadedconfigurable circuit may comprise: using a plurality of controlregisters, storing a thread identifier pool and a completion tablehaving a loop count of an active number of loop threads; and using athread control circuit, in response to receipt of an asynchronous fabricmessage returning a thread identifier to the thread identifier pool,decrementing the loop count and, when the loop count reaches zero,transmitting an asynchronous fabric completion message.

In any of the various representative embodiments, the method may furthercomprise: using the second, instruction and instruction index memory,providing a plurality of spoke instructions and data path configurationinstruction indices for selection of a current data path configurationinstruction for the configurable computation circuit.

In any of the various representative embodiments, the method may furthercomprise: using the second, instruction and instruction index memory,providing a plurality of spoke instructions and data path configurationinstruction indices for selection of a next data path configurationinstruction for a next configurable computation circuit.

In any of the various representative embodiments, the method may furthercomprise: using the second, instruction and instruction index memory,providing a plurality of spoke instructions and data path configurationinstruction indices for selection of a synchronous network output of theplurality of synchronous network outputs.

In any of the various representative embodiments, the method may furthercomprise: using a configuration memory multiplexer, providing a firstselection setting to select the current data path configurationinstruction using an instruction index from the second, instruction andinstruction index memory.

In any of the various representative embodiments, the method may furthercomprise: using a configuration memory multiplexer, providing a secondselection setting, the second setting different from the first setting,to select the current data path configuration instruction using aninstruction index from a master synchronous input.

In any of the various representative embodiments, the method may furthercomprise: using the second, instruction and instruction index memory,providing a plurality of spoke instructions and data path configurationinstruction indices for configuration of portions of the configurablecircuit independently from the current data path instruction.

In any of the various representative embodiments, the method may furthercomprise: using a configuration memory multiplexer, selecting a spokeinstruction and data path configuration instruction index of theplurality of spoke instructions and data path configuration instructionindices according to a modulo spoke count.

In any of the various representative embodiments, the method may furthercomprise: using a conditional logic circuit and depending upon an outputfrom the configurable computing circuit, modifying the next data pathinstruction or next data path instruction index.

In any of the various representative embodiments, the method may furthercomprise: using a conditional logic circuit and depending upon an outputfrom the configurable computing circuit, providing conditional branchingby modifying the next data path instruction or next data pathinstruction index.

In any of the various representative embodiments, the method may furthercomprise: enabling a conditional logic circuit; and using theconditional logic circuit and depending upon an output from theconfigurable computing circuit, providing conditional branching by ORingthe least significant bit of the next data path instruction with theoutput from the configurable computing circuit to designate the nextdata path instruction or data path instruction index.

In any of the various representative embodiments, the method may furthercomprise: using an input multiplexer, selecting the master synchronousinput. In any of the various representative embodiments, the method mayfurther comprise: using an output multiplexer, selecting an output fromthe configurable computing circuit.

In any of the various representative embodiments, the method may furthercomprise: using an asynchronous fabric state machine coupled to anasynchronous network input queue and to an asynchronous network outputqueue, decoding an input data packet received from the asynchronouspacket network and assembling an output data packet for transmission onthe asynchronous packet network.

In any of the various representative embodiments, the method may furthercomprise: using the synchronous network, providing a plurality of directpoint-to-point connections coupling adjacent configurable circuits ofthe array of the plurality of configurable circuits.

In any of the various representative embodiments, the method may furthercomprise: using the configurable circuit, providing a direct, passthrough connection between a plurality of input registers and aplurality of output registers. In any of the various representativeembodiments, the direct, pass through connection provides a direct,point-to-point connection for data transmission from a secondconfigurable circuit received on the synchronous network to a thirdconfigurable circuit transmitted on the synchronous network.

In any of the various representative embodiments, the method may furthercomprise: using the configurable computation circuit, performing atleast one integer or floating point operation selected from the groupconsisting of: signed and unsigned addition, absolute value, negate,logical NOT, add and negate, subtraction A-B, reverse subtraction B-A,signed and unsigned greater than, signed and unsigned greater than orequal to, signed and unsigned less than, signed and unsigned less thanor equal to, comparison of equal or not equal to, logical AND operation,logical OR operation, logical XOR operation, logical NAND operation,logical NOR operation, logical NOT XOR operation, logical AND NOToperation, logical OR NOT operation, and an interconversion betweeninteger and floating point.

In any of the various representative embodiments, the method may furthercomprise: using the configurable computation circuit, performing atleast one integer or floating point operation selected from the groupconsisting of: multiply, shift, pass an input, signed and unsignedmultiply, signed and unsigned shift right, signed and unsigned shiftleft, bit order reversal, a permutation, an interconversion betweeninteger and floating point, and combinations thereof.

In any of the various representative embodiments, the method may furthercomprise: using a dispatch interface circuit, receiving a workdescriptor packet over the first interconnection network, and inresponse to the work descriptor packet, to generate one or more data andcontrol packets to the plurality of configurable circuits to configurethe plurality of configurable circuits for execution of a selectedcomputation.

In any of the various representative embodiments, the method may furthercomprise: using a flow control circuit, generating a stop signal when apredetermined threshold has been reached in the asynchronous networkoutput queue. In any of the various representative embodiments, inresponse to the stop signal, each asynchronous network output queuestops outputting data packets on the asynchronous packet network. In anyof the various representative embodiments, in response to the stopsignal, each configurable computation circuit stops executing uponcompletion of its current instruction.

In any of the various representative embodiments, the method may furthercomprise: coupling a first plurality of configurable circuits of thearray of a plurality of configurable circuits in a first predeterminedsequence through the synchronous network to form a first synchronousdomain; and coupling a second plurality of configurable circuits of thearray of a plurality of configurable circuits are coupled in a secondpredetermined sequence through the synchronous network to form a secondsynchronous domain.

In any of the various representative embodiments, the method may furthercomprise: generating a continuation message from the first synchronousdomain to the second synchronous domain for transmission through theasynchronous packet network.

In any of the various representative embodiments, the method may furthercomprise: generating a completion message from the second synchronousdomain to the first synchronous domain for transmission through theasynchronous packet network.

In any of the various representative embodiments, the method may furthercomprise storing a completion table having a first, data completioncount in the plurality of control registers.

In any of the various representative embodiments, the method may furthercomprise: storing the completion table having a second, iteration countin the plurality of control registers.

In any of the various representative embodiments, the method may furthercomprise: storing a loop table having a plurality of thread identifiersin the plurality of control registers and, for each thread identifier,storing a next thread identifier for execution following execution of acurrent thread.

In any of the various representative embodiments, the method may furthercomprise: storing in the loop table in the plurality of controlregisters, an identification of a first iteration and an identificationof a last iteration.

In any of the various representative embodiments, the method may furthercomprise: using the control circuit, queueing a thread for executionwhen, for its thread identifier, its completion count has decremented tozero.

In any of the various representative embodiments, the method may furthercomprise: using the control circuit, queueing a thread for executionwhen, for its thread identifier, its completion count has decremented tozero and its thread identifier is the next thread.

In any of the various representative embodiments, the method may furthercomprise: using the control circuit, queueing a thread for executionwhen, for its thread identifier, its completion count indicatescompletion of any data dependencies. In any of the variousrepresentative embodiments, the completion count may indicate apredetermined number of completion messages to be received, per selectedthread of a plurality of threads, prior to execution of the selectedthread.

In any of the various representative embodiments, the method may furthercomprise: storing a completion table, in the plurality of controlregisters, having a plurality of types of thread identifiers, with eachtype of thread identifier indicating a loop level for loop and nestedloop execution.

In any of the various representative embodiments, the method may furthercomprise: storing, in the plurality of control registers, a completiontable having a loop count of an active number of loop threads, andwherein in response to receipt of an asynchronous fabric messagereturning a thread identifier to a thread identifier pool, using thecontrol circuit, decrementing the loop count and, when the loop countreaches zero, transmitting an asynchronous fabric completion message.

In any of the various representative embodiments, the method may furthercomprise: storing a top of thread identifiers stack in the plurality ofcontrol registers to allow each type of thread identifier access toprivate variables for a selected loop.

In any of the various representative embodiments, the method may furthercomprise: using a continuation queue, storing one or more threadidentifiers for computation threads having completion counts allowingexecution but do not yet have an assigned thread identifier.

In any of the various representative embodiments, the method may furthercomprise: using a reenter queue, storing one or more thread identifiersfor computation threads having completion counts allowing execution andhaving an assigned thread identifier.

In any of the various representative embodiments, the method may furthercomprise: executing any thread having a thread identifier in the reenterqueue prior to execution of any thread having a thread identifier in thecontinuation queue.

In any of the various representative embodiments, the method may furthercomprise: executing any thread having a thread identifier in a priorityqueue prior to execution of any thread having a thread identifier in thecontinuation queue or in the reenter queue.

In any of the various representative embodiments, the method may furthercomprise: executing any thread in a run queue upon an occurrence of aspoke count for the thread identifier.

In any of the various representative embodiments, the method may furthercomprise: using a control circuit, self-scheduling a computation threadfor execution.

In any of the various representative embodiments, the method may furthercomprise: using the conditional logic circuit, branching to a different,second next instruction for execution by a next configurable circuit.

In any of the various representative embodiments, the method may furthercomprise: using the control circuit, ordering computation threads forexecution.

In any of the various representative embodiments, the method may furthercomprise: using the control circuit, ordering loop computation threadsfor execution.

In any of the various representative embodiments, the method may furthercomprise: using the control circuit, commencing execution of computationthreads in response to one or more completion signals from datadependencies.

Numerous other advantages and features of the present invention willbecome readily apparent from the following detailed description of theinvention and the embodiments thereof, from the claims and from theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects, features and advantages of the present invention will bemore readily appreciated upon reference to the following disclosure whenconsidered in conjunction with the accompanying drawings, wherein likereference numerals are used to identify identical components in thevarious views, and wherein reference numerals with alphabetic charactersare utilized to identify additional types, instantiations or variationsof a selected component embodiment in the various views, in which:

FIG. 1 is a block diagram of a representative first embodiment of ahybrid computing system.

FIG. 2 is a block diagram of a representative second embodiment of ahybrid computing system.

FIG. 3 is a block diagram of a representative third embodiment of ahybrid computing system.

FIG. 4 is a block diagram of a representative embodiment of a hybridthreading fabric having configurable computing circuitry coupled to afirst interconnection network.

FIG. 5 is a high-level block diagram of a portion of a representativeembodiment of a hybrid threading fabric circuit cluster.

FIG. 6 is a high-level block diagram of a second interconnection networkwithin a hybrid threading fabric circuit cluster.

FIG. 7 is a detailed block diagram of a representative embodiment of ahybrid threading fabric circuit cluster.

FIG. 8 is a detailed block diagram of a representative embodiment of ahybrid threading fabric configurable computing circuit (tile).

FIGS. 9, 9A and 9B, in which FIG. 9 is divided into FIGS. 9A and 9B (andcollectively referred to as FIG. 9 ) are collectively a detailed blockdiagram of a representative embodiment of a hybrid threading fabricconfigurable computing circuit (tile).

FIG. 10 is a detailed block diagram of a representative embodiment of amemory control circuit of a hybrid threading fabric configurablecomputing circuit (tile).

FIG. 11 is a detailed block diagram of a representative embodiment of athread control circuit of a hybrid threading fabric configurablecomputing circuit (tile).

FIG. 12 is a diagram of representative hybrid threading fabricconfigurable computing circuits (tiles) forming synchronous domains andrepresentative asynchronous packet network messaging.

FIG. 13 is a block diagram of a representative embodiment of a memoryinterface.

FIG. 14 is a block diagram of a representative embodiment of a dispatchinterface.

FIG. 15 is a block diagram of a representative embodiment of an optionalfirst network interface.

FIG. 16 is a diagram of representative hybrid threading fabricconfigurable computing circuits (tiles) forming synchronous domains andrepresentative asynchronous packet network messaging for performance ofa computation by a hybrid threading fabric circuit cluster.

FIGS. 17A, 17B, and 17C (collectively FIG. 17 ) is a flow chart ofrepresentative asynchronous packet network messaging and execution byhybrid threading fabric configurable computing circuits (tiles) forperformance of the computation of FIG. 16 by a hybrid threading fabriccircuit cluster.

FIG. 18 is a diagram of representative hybrid threading fabricconfigurable computing circuits (tiles) forming synchronous domains andrepresentative asynchronous packet network messaging for performance ofa computation by a hybrid threading fabric circuit cluster.

FIGS. 19A, and 19B (collectively FIG. 19 ) is a flow chart ofrepresentative asynchronous packet network messaging and execution byhybrid threading fabric configurable computing circuits (tiles) forperformance of the computation of FIG. 18 by a hybrid threading fabriccircuit cluster.

FIG. 20 is a diagram of representative hybrid threading fabricconfigurable computing circuits (tiles) forming synchronous domains andrepresentative asynchronous packet network messaging for performance ofa loop in a computation by a hybrid threading fabric circuit cluster.

FIG. 21 is a flow chart of representative asynchronous packet networkmessaging and execution by hybrid threading fabric configurablecomputing circuits (tiles) for performance of the loop in a computationof FIG. 20 by a hybrid threading fabric circuit cluster.

FIG. 22 is a block diagram of a representative flow control circuit.

FIG. 23 is a diagram of representative hybrid threading fabricconfigurable computing circuits (tiles) forming synchronous domains andrepresentative asynchronous packet network messaging and synchronousmessaging for performance of a loop in a computation by a hybridthreading fabric circuit cluster.

FIG. 24 is a block and circuit diagram of a representative embodiment ofconditional branching circuitry.

DETAILED DESCRIPTION OF REPRESENTATIVE EMBODIMENTS

While the present invention is susceptible of embodiment in manydifferent forms, there are shown in the drawings and will be describedherein in detail specific exemplary embodiments thereof, with theunderstanding that the present disclosure is to be considered as anexemplification of the principles of the invention and is not intendedto limit the invention to the specific embodiments illustrated. In thisrespect, before explaining at least one embodiment consistent with thepresent invention in detail, it is to be understood that the inventionis not limited in its application to the details of construction and tothe arrangements of components set forth above and below, illustrated inthe drawings, or as described in the examples. Methods and apparatusesconsistent with the present invention are capable of other embodimentsand of being practiced and carried out in various ways. Also, it is tobe understood that the phraseology and terminology employed herein, aswell as the abstract included below, are for the purposes of descriptionand should not be regarded as limiting.

FIGS. 1, 2 and 3 are block diagrams of representative first, second, andthird embodiments of a hybrid computing system 100A, 100B, 100C(collectively referred to as a system 100). FIG. 4 is a block diagram ofa representative embodiment of a hybrid threading fabric (“HTF”) 200having configurable computing circuitry coupled to a firstinterconnection network 150. FIG. 5 is a high-level block diagram of aportion of a representative embodiment of a hybrid threading fabriccluster 205. FIG. 6 is a high-level block diagram of a secondinterconnection network within a hybrid threading fabric cluster 205.FIG. 7 is a detailed block diagram of a representative embodiment of ahybrid threading fabric (HTF) cluster 205.

FIG. 8 is a high-level block diagram of a representative embodiment of ahybrid threading fabric configurable computing circuit 210, referred toas a “tile” 210. FIGS. 9, 9A and 9B, in which FIG. 9 is divided intoFIGS. 9A and 9B (and collectively referred to as FIG. 9 ) arecollectively detailed block diagram of a representative embodiment of ahybrid threading fabric configurable computing circuit 210A, referred toas a “tile” 210A, as a particular representative instantiation of a tile210. Unless specifically referred to as a tile 210A, reference to a tile210 shall mean and refer, individually and collectively, to a tile 210and tile 210A. The hybrid threading fabric configurable computingcircuit 210 is referred to as a “tile” 210 because all such hybridthreading fabric configurable computing circuits 210, in arepresentative embodiment, are identical to each other and can bearrayed and connected in any order, i.e., each hybrid threading fabricconfigurable computing circuit 210 can be “tiled” to form a hybridthreading fabric cluster 205.

Referring to FIGS. 1-9 , a hybrid computing system 100 includes a hybridthreading processor (“HTP”) 300, which is coupled through a firstinterconnection network 150 to one or more hybrid threading fabric(“HTF”) circuits 200. It should be understood that term “fabric”, asused herein, means and includes an array of computing circuits, which inthis case are reconfigurable computing circuits. FIGS. 1, 2, and 3 showdifferent system 100A, 100B, and 100C arrangements which includeadditional components forming comparatively larger and smaller systems100, any and all of which are within the scope of the disclosure. Asshown in FIGS. 1 and 2 , which may each be an arrangement suitable for asystem-on-a-chip (“SOC”), for example and without limitation, a hybridcomputing system 100A, 100B, in various combinations as illustrated, mayalso include, optionally, a memory controller 120 which may be coupledto a memory 125 (which also may be a separate integrated circuit), anyof various communication interfaces 130 (such as a PCIe communicationinterface), one or more host processor(s) 110, and a host interface(“HIF”) 115. As shown in FIG. 3 , which may each be an arrangementsuitable for a “chiplet” configuration on a common substrate 101, forexample and without limitation, a hybrid computing system 100C may alsoinclude, optionally, a communication interface 130, with or withoutthese other components. Any and all of these arrangements are within thescope of the disclosure, and collectivey are referred to herein as asystem 100. Any of these hybrid computing systems 100 also may beconsidered a “node”, operating under a single operating system (“OS”),and may be coupled to other such local and remote nodes as well.

Hybrid threading, as used herein, refers to the capability to spawnmultiple fibers and threads of computation across different,heterogeneous types of processing circuits (hardware), such as acrossHTF circuits 200 (as a reconfigurable computing fabric) and across aprocessor, such as a RISC-V processor. Hybrid threading also refers to aprogramming language/style in which a thread of work transitions fromone compute element to the next to move the compute to where the data islocated, which is also implemented in representative embodiments. Alsoin a representative embodiment, the HTP 300 is a RISC-V ISA basedmulti-threaded processor. The HTP 300 provides barrel-style, round-robininstantaneous thread switching to maintain a high instruction-per-clockrate. A host processor 110 is typically a multi-core processor, whichmay be embedded within the hybrid computing system 100, or which may bean external host processor coupled into the hybrid computing system 100via a communication interface 130, such as a PCIe-based interface. Theseprocessors, such as the HTP 300 and the one or more host processor(s)110, are described in greater detail below.

The memory controller 120 may be implemented as known or becomes knownin the electronic arts. Alternatively, in a representative embodiment,the memory controller 120 may be implemented as described in the relatedapplications. The memory 125 also may be implemented as known or becomesknown in the electronic arts, and as described in greater detail below.

The HIF 115, for the purposes herein, provides for a host processor 110to send work to the HTP 300 and the HTF circuits 200, and for the HTP300 to send work to the HTF circuits 200, both as work descriptorpackets transmitted over the first interconnection network 150. The HIF115 includes a dispatch circuit and queue (abbreviated “dispatch queue”105), which also provides management functionality for monitoring theload provided to and resource availability of the HTF circuits 200and/or HTP 300. When resources are available on the HTF circuits 200and/or HTP 300, the dispatch queue 105 determines the HTF circuit 200and/or HTP 300 resource that is least loaded. In the case of multipleHTF circuit clusters 205 with the same or similar work loading, itchooses an HTF circuit cluster 205 that is currently executing the samekernel if possible (to avoid having to load or reload a kernelconfiguration). Similar functionality of the HIF 115 may also beincluded in an HTP 300, for example, particularly for system 100arrangements which may not include a separate HIF 115. Other HIF 115functions are described in greater detail below. An HIF 115 may beimplemented as known or becomes known in the electronic arts, e.g., asone or more state machines with registers (forming FIFOs, queues, etc.).

The first interconnection network 150 is a packet-based communicationnetwork providing data packet routing between and among the HTF circuits200, the hybrid threading processor 300, and the other optionalcomponents such as the memory controller 120, a communication interface130, and a host processor 110. The first interconnection network 150 istypically embodied as a plurality of crossbar switches having a foldedclos configuration, and typically a mesh network for additionalconnections, depending upon the system 100 embodiment. For purposes ofthe present disclosure, the first interconnection network 150 forms partof an asynchronous switching fabric (“AF”), meaning that a data packetmay be routed along any of various paths, such that the arrival of anyselected data packet at an addressed destination may occur at any of aplurality of different times, depending upon the routing. This is incontrast with the synchronous mesh communication network 275 of thesecond interconnection network 250 discussed in greater detail below.

A HTF circuit 200, in turn, typically comprises a plurality of HTFcircuit clusters 205, with each HTF circuit cluster 205 coupled to thefirst interconnection network 150 for data packet communication. EachHTF circuit cluster 205 may operate independently from each of the otherHTF circuit clusters 205. Each HTF circuit cluster 205, in turn,comprises an array of a plurality of HTF reconfigurable computingcircuits 210, which are referred to equivalently herein as “tiles” 210,and a second interconnection network 250. The tiles 210 are embedded inor otherwise coupled to the second interconnection network 250, whichcomprises two different types of networks, discussed in greater detailbelow. In a representative embodiment, each HTF circuit cluster 205 alsocomprises a memory interface 215, an optional first network interface220 (which provides an interface for coupling to the firstinterconnection network 150), and a dispatch interface 225. The variousmemory interfaces 215, the dispatch interface 225, and the optionalfirst network interface 220 may be implemented using any appropriatecircuitry, such as one or more state machine circuits, to perform thefunctionality specified in greater detail below.

As an overview, the HTF circuit 200 is a course grained reconfigurablecompute fabric comprised of interconnected compute tiles 210. The tiles210 are interconnected with a synchronous fabric referred to as thesynchronous mesh communication network 275, allowing data to traversefrom one tile 210 to another tile 210 without queuing. This synchronousmesh communication network 275 allows many tiles 210 to be pipelinedtogether to produce a continuous data flow through arithmeticoperations, and each such pipeline of tiles 210 connected through thesynchronous mesh communication network 275 for performance of one ormore threads of computation is referred to herein as a “synchronousdomain”, which may have series connections, parallel connections, andpotentially branching connections as well. The first tile 210 of asynchronous domain is referred to herein as a “base” tile 210.

The tiles 210 are also interconnected with an asynchronous fabricreferred to as an asynchronous packet network 265 that allowssynchronous domains of compute to be bridged by asynchronous operations,with all packets on the asynchronous packet network 265 capable of beingcommunicated in a single clock cycle in representative embodiments.These asynchronous operations include initiating synchronous domainoperations, transferring data from one synchronous domain to another,accessing system memory 125 (read and write), and performing branchingand looping constructs. Together, the synchronous and asynchronousfabrics allow the tiles 210 to efficiently execute high level languageconstructs. The asynchronous packet network 265 differs from the firstinterconnection network 150 in many ways, including requiring lessaddressing, being a single channel, being queued with a depth-basedbackpressure, and utilizing packed data operands, such as with a datapath of 128 bits, for example and without limitation. It should be notedthat the internal data paths of the various tiles 210 are also 128 bits,also for example and without limitation. Examples of synchronousdomains, and examples of synchronous domains communicating with eachother over the asynchronous packet network 265, are illustrated in FIGS.16, 18, 20 , for example and without limitation.

In a representative embodiment, under most circumstances, thread (e.g.,kernel) execution and control signaling are separated between these twodifferent networks, with thread execution occurring using thesynchronous mesh communication network 275 to form a plurality ofsynchronous domains of the various tiles 210, and control signalingoccurring using messaging packets transmitted over the asynchronouspacket network 265 between and among the various tiles 210. For example,the plurality of configurable circuits are adapted to perform aplurality of computations using the synchronous mesh communicationnetwork 275 to form a plurality of synchronous domains, and theplurality of configurable circuits are further adapted to generate andtransmit a plurality of control messages over the asynchronous packetnetwork 265, with the plurality of control messages comprising one ormore completion messages and continue messages, for example and withoutlimitation.

In a representative embodiment, the second interconnection network 250typically comprises two different types of networks, each providing datacommunication between and among the tiles 210, a first, asynchronouspacket network 265 overlaid or combined with a second, synchronous meshcommunication network 275, as illustrated in FIGS. 6 and 7 . Theasynchronous packet network 265 is comprised of a plurality of AFswitches 260, which are typically implemented as crossbar switches(which may or may not additionally or optionally have a Clos or FoldedClos configuration, for example and without limitation), and a pluralityof communication lines (or wires) 280, 285, connecting the AF switches260 to the tiles 210, providing data packet communication between andamong the tiles 210 and the other illustrated components discussedbelow. The synchronous mesh communication network 275 provides aplurality of direct (i.e., unswitched, point-to-point) connections overcommunication lines (or wires) 270 between and among the tiles 210,which may be register-staged at the inputs and outputs of the tile 210,but otherwise without queuing between tiles 210, and without requiringformation of a data packet. (In FIG. 6 , to better illustrate theoverlaying of the two networks with tiles 210 embedded in the secondinterconnection network 250, the tiles 210 are represented as thevertices of the synchronous mesh communication network 275, and the AFswitches 260 are illustrated as “Xs”, as indicated.)

Referring to FIG. 8 , a tile 210 comprises one or more configurablecomputation circuits 155, control circuitry 145, one or more memories325, a configuration memory (e.g., RAM) 160, synchronous networkinput(s) 135 (coupled to the synchronous mesh communication network275), synchronous network output(s) 170 (also coupled to the synchronousmesh communication network 275), asynchronous (packet) network input(s)140 (coupled to the asynchronous packet network 265), and asynchronous(packet) network output(s) 165 (also coupled to the asynchronous packetnetwork 265). Each of these various components are shown coupled to eachother, in various combinations as illustrated, over busses 180, 185.Those having skill in the electronic arts will recognize that fewer ormore components may be included in a tile 210, along with any of variouscombinations of couplings, any and all of which are consideredequivalent and within the scope of the disclosure.

Representative examples of each of these various components areillustrated and discussed below with reference to FIG. 9 . For example,in a representative embodiment, the one or more configurable computationcircuits 155 are embodied as a multiply and shift operation circuit (“MSOp”) 305 and an Arithmetic, Logical and Bit Operation circuit (“ALB Op”)310, with associated configuration capabilities, such as throughintermediate multiplexers 365, and associated registers, such asregisters 312, for example and without limitation. Also in arepresentative embodiment, the one or more configurable computationcircuits 155 may include a write mask generator 375 and conditional(branch) logic circuitry 370, also for example and without limitation.Also in a representative embodiment, control circuitry 145 may includememory control circuitry 330, thread control circuitry 335, and controlregisters 340, such as those illustrated for a tile 210A, for exampleand without limitation. Continuing with the examples, synchronousnetwork input(s) 135 may be comprised of input registers 350 and inputmultiplexers 355, synchronous network output(s) 170 may be comprised ofoutput registers 380 and output multiplexers 395, asynchronous (packet)network input(s) 140 may be comprised of AF input queues 360 andasynchronous (packet) network output(s) 165 may be comprised of AFoutput queues 390, and may also include or share an AF message statemachine 345.

Significantly, and as discussed in greater detail below, theconfiguration memory (e.g., RAM) 160 is comprised of configurationcircuitry (such as configuration memory multiplexer 372) and twodifferent configuration stores which perform different configurationfunctions, a first instruction RAM 315 (which is used to configure theinternal data path of a tile 210) and a second instruction (and index)RAM 320, referred to herein as a “spoke” RAM 320 (which is used formultiple purposes, including to configure portions of a tile 210 whichare independent from a current instruction, to select a currentinstruction and an instruction of a next tile 210, and to select amaster synchronous input, among other things, all as discussed ingreater detail below).

As illustrated in FIG. 8 and FIG. 9 , the communication lines (or wires)270 are illustrated as communication lines (or wires) 270A and 270B,such that communication lines (or wires) 270A are the “inputs” (inputcommunication lines (or wires)) feeding data into the input registers350, and the communication lines (or wires) 270B are the “outputs”(output communication lines (or wires)) moving data from the outputregisters 380. In a representative embodiment, there are a plurality ofsets or busses of communication lines (or wires) 270 into and out ofeach tile 210, from and to each adjacent tile (e.g., synchronous meshcommunication network 275 up link, down link, left link, and rightlink), and from and to other components for distribution of varioussignals, such as data write masks, stop signals, and instructions orinstruction indices provided from one tile 210 to another tile 210, asdiscussed in greater detail below. Alternatively, and not separatelyillustrated, there may also be various dedicated communication lines,such as for asserting a stop signal, such that a stop signal generatedfrom any tile 210 in a HTF circuit cluster 205 can be received promptly,in a limited number of clock cycles, by all other tiles 210 in the HTFcircuit cluster 205.

It should be noted that there are various fields in the various sets orbusses of communication lines forming the synchronous mesh communicationnetwork 275. For example, FIG. 8 and FIG. 9 illustrate four busses ofincoming and outgoing communication lines (or wires) 270A and 270B,respectively. Each one of these sets of communication lines (or wires)270A and 270B may carry different information, such as data, aninstruction index, control information, and thread information (such asTID, XID, loop dependency information, write mask bits for selection ofvalid bits, etc.). One of the inputs 270A may also be designated as amaster synchronous input, including input internal to a tile 210 (fromfeedback of an output), which can vary for each time slice of a tile210, which may have the data for an instruction index for that tile 210of a synchronous domain, for example and without limitation, discussedin greater detail below.

In addition, as discussed in greater detail below, for any inputreceived on the synchronous mesh communication network 275 and held inone or more input registers 350 (of the synchronous network input(s)135), each tile 210 may transfer that input directly to one or moreoutput registers 380 (of the synchronous network output(s) 170) foroutput (typically on a single clock cycle) to another location of thesynchronous mesh communication network 275, thereby allowing a firsttile 210 to communicate, via one or more intermediate, second tiles 210,with any other third tile 210 within the HTF circuit cluster 205. Thissynchronous mesh communication network 275 enables configuration (andreconfiguration) of a statically scheduled, synchronous pipeline betweenand among the tiles 210, such that once a thread is started along aselected data path between and among the tiles 210, as a synchronousdomain, completion of the data processing will occur within a fixedperiod of time. In addition, the synchronous mesh communication network275 serves to minimize the number of any required accesses to memory125, as accesses to memory 125 may not be required to complete thecomputations for that thread be performed along the selected data pathbetween and among the tiles 210.

In the asynchronous packet network 265, each AF switch 260 is typicallycoupled to a plurality of tiles 210 and to one or more other AF switches260, over communication lines (or wires) 280. In addition, one or moreselected AF switches 260 are also coupled (over communication lines (orwires) 285) to one or more of the memory interface 215, the optionalfirst network interface 220, and the dispatch interface 225. Asillustrated, as an example and without limitation, the HTF circuitcluster 205 includes a single dispatch interface 225, two memoryinterfaces 215, and two optional first network interfaces 220. Also asillustrated, as an example and without limitation, in addition to beingcoupled to the other AF switches 260, one of the AF switches 260 isfurther coupled to a memory interface 215, to an optional first networkinterface 220, and to the dispatch interface 225, while another one ofthe AF switches 260 is further coupled to a memory interface 215 and tothe optional first network interface 220.

Depending upon the selected embodiment, each of the memory interfaces215 and the dispatch interface 225 may also be directly connected to thefirst interconnection network 150, with capability for receiving,generating, and transmitting data packets over both the firstinterconnection network 150 and the asynchronous packet network 265, anda first network interface 220 is not utilized or included in HTF circuitclusters 205. For example, the dispatch interface 225 may be utilized byany of the various tiles 210 for transmission of a data packet to andfrom the first interconnection network 150. In other embodiments, any ofthe memory interfaces 215 and the dispatch interface 225 may utilize thefirst network interface 220 for receiving, generating, and transmittingdata packets over the first interconnection network 150, such as to usethe first network interface 220 to provide additional addressing neededfor the first interconnection network 150.

Those having skill in the electronic arts will recognize that theconnections between and among the AF switches 260, the tiles 210, theoptional first network interfaces 220, the memory interfaces 215, andthe dispatch interface 225 may occur in any selected combination, withany selected number of components, and with any all such componentselections and combinations considered equivalent and within the scopeof the disclosure. Those having skill in the electronic arts willrecognize that the division of the HTF circuit 200 into a plurality ofHTF clusters 205 is not required, and merely provides a conceptualdivision for ease of describing the various components and theconnections between and among the various components. For example, whilea HTF circuit cluster 205 is illustrated as having sixteen tiles 210,with four AF switches 260, a single dispatch interface 225, two memoryinterfaces 215, and two first network interfaces 220 (optional), more orfewer of any of these components may be included in either or both a HTFcircuit cluster 205 or a HTF circuit 200, and as described in greaterdetail below, for any selected embodiment, an HTF circuit cluster 205may be partitioned to vary the number and type of component which may beactive (e.g., powered on and functioning) at any selected time.

The synchronous mesh communication network 275 allows multiple tiles 210to be pipelined without the need for data queuing. All tiles 210 thatparticipate in a synchronous domain act as a single pipelined data path.The first tile of such a sequence of tiles 210 forming a singlepipelined data path is referred to herein as a “base” tile 210 of asynchronous domain, and such a base tile 210 initiates a thread of workthrough the pipelined tiles 210. The base tile 210 is responsible forstarting work on a predefined cadence referred to herein as the “spokecount”. As an example, if the spoke count is three, then the base tile210 can initiate work every third clock. It should also be noted thatthe computations within each tile 210 can also be pipelined, so thatparts of different instructions can be performed while otherinstructions are executing, such as data being input for a nextoperation while a current operation is executing.

Each of the tiles 210, the memory interfaces 215, and the dispatchinterface 225 has a distinct or unique address (e.g., as a 5-bit wideendpoint ID), as a destination or end point, within any selected HTFcircuit cluster 205. For example, the tiles 210 may have endpoint IDs of0-15, memory interfaces 215 (0 and 1) may have endpoint IDs of 20 and21, and dispatch interface 225 may have endpoint ID of 18 (with noaddress being provided to the optional first network interface 220,unless it is included in a selected embodiment). The dispatch interface225 receives a data packet containing work to be performed by one ormore of the tiles 210, referred to as a work descriptor packet, whichhave been configured for various operations, as discussed in greaterdetail below. The work descriptor packet will have one or morearguments, which the dispatch interface 225 will then provide ordistribute to the various tiles, as a packet or message (AF message)transmitted through the AF switches 260, to the selected, addressedtiles 210, and further, will typically include an identification of aregion in tile memory 325 to store the data (argument(s)), and a threadidentifier (“ID”) utilized to track and identify the associatedcomputations and their completion.

Messages are routed from source endpoint to destination endpoint throughthe asynchronous packet network 265. Messages from different sources tothe same destination take different paths and may encounter differentlevels of congestion. Messages may arrive in a different order than whenthey are sent out. The messaging mechanisms are constructed to workproperly with non-deterministic arrival order.

FIG. 13 is a block diagram of a representative embodiment of a memoryinterface 215. Referring to FIG. 13 , each memory interface 215comprises a state machine (and other logic circuitry) 480, one or moreregisters 485, and optionally one or more queues 474. The state machine480 receives, generates, and transmits data packets on the asynchronouspacket network 265 and the first interconnection network 150. Theregisters 485 store addressing information, such as virtual addresses oftiles 210, physical addresses within a given node, and various tables totranslate virtual addresses to physical addresses. The optional queues474 store messages awaiting transmission on the first interconnectionnetwork 150 and/or the asynchronous packet network 265.

The memory interface 215 allows the tiles 210 within a HTF circuitcluster 205 to make requests to the system memory 125, such as DRAMmemory. The memory request types supported by the memory interface 215are loads, stores and atomics. From the memory interface 215perspective, a load sends an address to memory 125 and data is returned.A write sends both an address and data to memory 125 and a completionmessage is returned. An atomic operation sends an address and data tomemory 125, and data is returned. It should be noted that an atomic thatjust receives data from memory (i.e. fetch-and-increment) would behandled as a load request by the memory interface 215. All memoryinterface 215 operations require a single 64-bit virtual address. Thedata size for an operation is variable from a single byte to 64 bytes.Larger data payload sizes are more efficient for the device and can beused; however, the data payload size will be governed by the ability ofthe high level language compiler to detect access to large blocks ofdata.

FIG. 14 is a block diagram of a representative embodiment of a dispatchinterface 225. Referring to FIG. 14 , a dispatch interface 225 comprisesa state machine (and other logic circuitry) 470, one or more registers475, and one or more dispatch queues 472. The state machine 470receives, generates, and transmits data packets on the asynchronouspacket network 265 and the first interconnection network 150. Theregisters 475 store addressing information, such as virtual addresses oftiles 210, and a wide variety of tables tracking the configurations andworkloads distributed to the various tiles, discussed in greater detailbelow. The dispatch queues 472 store messages awaiting transmission onthe first interconnection network 150 and/or the asynchronous packetnetwork 265.

As mentioned above, the dispatch interface 225 receives work descriptorcall packet (messages), such as from the host interface 115, over thefirst interconnection network 150. The work descriptor call packet willhave various information, such as a payload (e.g., a configuration,argument values, etc.), a call identifier (ID), and return information(for the provision of results to that endpoint, for example). Thedispatch interface 225 will create various AF data messages fortransmission over the asynchronous packet network 265 to the tiles 210,including to write data into memories 325, which tile 210 will be thebase tile 210 (a base tile ID, for transmission of an AF completionmessage), a thread ID (thread identifier or “TID”), and will send acontinuation message to the base tile 210 (e.g., with completion andother counts for each TID), so that the base tile 210 can commenceexecution once it has received sufficient completion messages. Thedispatch interface 225 maintains various tables in registers 475 totrack what has been transmitted to which tile 210, per thread ID andXID. As results are generated or executions completed, the dispatchinterface 225 will receive AF data messages (indicating complete andwith data) or AF completion messages (indicating completion but withoutdata). The dispatch interface 225 has also maintained various counts (inregisters 475) of the number of completion and data messages it willneed to receive to know that kernel execution has completed, and willthen assemble and transmit the work descriptor return data packets, withthe resulting data, a call ID, the return information (e.g., address ofthe requestor), via the first interconnection network 150, and frees theTID. Additional features and functionality of the dispatch interface 225are described in greater detail below.

It should be noted, as mentioned above, that multiple levels (ormultiple types) of TIDs may be and typically are utilized. For example,the dispatch interface 225 allocates a first type of TID, from a pool ofTIDs, which it transmits to a base tile 210. The base tile 210, in turn,may allocate additional TIDs, such as second and third types of TIDs,such as for tracking the threads utilized in loops and nested loops, forexample and without limitation. These different TIDs then can also beutilized to access variables which are private to a given loop. Forexample, a first type of TID may be used for an outer loop, and secondand third types of TIDs may be utilized to track iterations of nestedloops.

It should also be noted that a separate transaction ID is utilized fortracking various memory requests over the first interconnection network150.

FIG. 15 is a block diagram of a representative embodiment of an optionalfirst network interface. Referring to FIG. 15 , when included each firstnetwork interface 220 comprises a state machine (and other logiccircuitry) 490 and one or more registers 495. The state machine 490receives, generates, and transmits data packets on the asynchronouspacket network 265 and the first interconnection network 150. Theregisters 495 store addressing information, such as virtual addresses oftiles 210, physical addresses within a given node, and various tables totranslate virtual addresses to physical addresses.

Referring again to FIG. 9 , a representative HTF reconfigurablecomputing circuit (tile) 210A comprises at least one multiply and shiftoperation circuit (“MS Op”) 305, at least one Arithmetic, Logical andBit Operation circuit (“ALB Op”) 310, a first instruction RAM 315, asecond, instruction (and index) RAM 320 referred to herein as a “spoke”RAM 320, one or more tile memory circuits (or memory) 325 (illustratedas memory “0” 325A, memory “1” 325B, through memory “N” 325C, andindividually and collectively referred to as memory 325 or tile memory325). In addition, as previously mentioned, a representative tile 210Aalso typically includes input registers 350 and output registers 380coupled over communication lines (or wires) 270A, 270B to thesynchronous mesh communication network 275, and AF input queues 360 andAF output queues 390 coupled over the communication lines (or wires) 280of the asynchronous packet network 265 to the AF switches 260. Controlcircuits 145 are also typically included in a tile 210, such as memorycontrol circuitry 330, thread control circuitry 335, and controlregisters 340 illustrated for a tile 210A. For decoding and forpreparing (assembling) data packets received from or provided to theasynchronous packet network 265, an AF message state machine 345 is alsotypically included in a tile 210. As part of configurability of the tile210, one or more multiplexers are typically included, illustrated asinput multiplexer 355, output multiplexer 395, and one or moreintermediate multiplexer(s) 365 for selection of the inputs to the MS Op305 and the ALB Op 310. Optionally, other components may also beincluded in a tile 210, such as conditional (branch) logic circuit 370,write mask generator 375, and flow control circuit 385 (which isillustrated as included as part of the AF output queues 390, and whichmay be provided as a separate flow control circuit, equivalently). Thecapabilities of the MS Op 305 and the ALB Op 310 are described ingreater detail below.

The synchronous mesh communication network 275 transfers informationrequired for the synchronous domain to function. The synchronous meshcommunication network 275 includes the fields specified below. Inaddition, many of the parameters used in these fields are also stored inthe control registers 340, and are assigned to a thread to be executedin the synchronous domain formed by a plurality of tiles 210. Thespecified fields of the synchronous mesh communication network 275include:

1. Data, typically having a field width of 64 bits, and comprisingcomputed data being transferred from one tile 210 to the next tile 210in a synchronous domain.

2. An instruction RAM 315 address, abbreviated “INSTR”, typically havinga field width of 8 bits, and comprising an instruction RAM 315 addressfor the next tile 210. The base tile 210 specifies the instruction RAM315 address for the first tile 210 in the domain. Subsequent tiles 210can pass the instruction unmodified, or can conditionally change theinstruction for the next tile 210 allowing conditional execution (i.e.if-then-else or switch statements), described in greater detail below.

3. A thread identifier, referred to herein as a “TID”, typically havinga field width of 8 bits, and comprising a unique identifier for threadsof a kernel, with a predetermined number of TIDs (a “pool of TIDs”)stored in the control registers 340 and potentially available for use bya thread (if not already in use by another thread). The TID is allocatedat a base tile 210 of a synchronous domain and can be used as a readindex into the tile memory 325. The TID can be passed from onesynchronous domain to another through the asynchronous packet network265. As there are a finite number of TIDs available for use, to performother functions or computations, eventually the TID should be freed backto the allocating base tile's TID pool for subsequent reuse. The freeingis accomplished using an asynchronous fabric message transmitted overthe asynchronous packet network 265.

4. A transfer identifier referred to as an “XID”, typically having afield width of 8 bits, and comprising a unique identifier fortransferring data from one synchronous domain to another, with apredetermined number of XIDs (a “pool of XIDs”) stored in the controlregisters 340 and potentially available for use by a thread (if notalready in use by another thread). The transfer may be a direct write ofdata from one domain to another, as an “XID_WR”, or it may be the resultof a memory 125 read (as an “XID_RD”) where the source domain sends avirtual address to memory 125 and the destination domain receives memoryread data. The XID_WR is allocated at the base tile 210 of the sourcedomain. The XID_WR in the source domain becomes the XID_RD in thedestination domain. The XID_WR can be used as a write index for tilememory 325 in the destination domain. XID_RD is used in the destinationdomain as a tile memory 325 read index. As there are a finite number ofXIDs available for use, to perform other functions or computations,eventually the XID should be freed back to the allocating base tile'sXID pool for subsequent reuse. The destination domain should free theXID by sending an asynchronous message to the source domain's base tile210, also over the asynchronous packet network 265.

The synchronous mesh communication network 275 provides both data andcontrol information. The control information (INSTR, XID, TID) is usedto setup the data path, and the DATA field can be selected as a sourcefor the configured data path. Note that the control fields are requiredmuch earlier (to configure the data path) than the data field. In orderto minimize the synchronous domain pipeline delay through a tile 210,the control information arrives at a tile 210 a few clock cycles earlierthan the data.

A particularly inventive feature of the architecture of the HTF circuit200 and its composite HTF circuit clusters 205 and their composite tiles210 is the use of two different configuration RAMs, the instruction RAM315 for data path configuration, and the spoke RAM 320 for multipleother functions, including configuration of portions of a tile 210 whichare independent from any selected or given data path, selection of datapath instructions from the instruction RAM 315, selection of the mastersynchronous input (among the available inputs 270A) for each clockcycle, and so on. As discussed in greater detail below, this novel useof both an instruction RAM 315 and an independent spoke RAM 320 enables,among other things, dynamic self-configuration and self-reconfigurationof the HTF circuit cluster 205 and of the HTF circuit 200 as a whole.

Each tile has an instruction RAM 315 that contains configurationinformation to setup the tile 210 data path for a specific operation,i.e., data path instructions that determine, for example, whether amultiplication, a shift, an addition, etc. will be performed in a giventime slice of the tile 210, and using which data (e.g., data from amemory 325, or data from an input register 350). The instruction RAM 315has multiple entries to allow a tile 210 to be time sliced, performingmultiple, different operations in a pipelined synchronous domain, withrepresentative pipeline sections 304, 306, 307, 308, and 309 of a tile210 illustrated in FIG. 9 . Any given instruction may also designatewhich inputs 270A will have the data and/or control information to beutilized by that instruction. Additionally, each time slice couldconditionally perform different instructions depending on previous tile210 time slice data dependent conditional operations, discussed withreference to FIG. 24 . The number of entries within the instruction RAM315 typically will be on the order of 256. The number may changedepending on the experience gained from porting kernels to the HTF 200.

The supported instruction set should match the needs of the targetapplications, such as for applications having data types of 32 and64-bit integer and floating point values. Additional applications suchas machine learning, image analysis, and 5G wireless processing may beperformed using the HTF 200. This total set of applications would need16, 32 and 64-bit floating point, and 8, 16, 32 and 64-bit integer datatypes. The supported instruction set needs to support these data typesfor load, store and arithmetic operations. The operations supported needto allow a compiler to efficiently map high level language source totile 210 instructions. In a representative embodiment, the tiles 210support the same instruction set as a standard high performanceprocessor, including single instruction multiple data (SIMD) instructionvariants.

The spoke RAM 320 has multiple functions, and in representativeembodiments, one of those functions is to be utilized to configure partsof (a time slice of) the tile 210 that is or are independent of thecurrent instruction for the data path, i.e., the tile 210 configurationsheld in the spoke RAM 320 can be used to configure invariant parts ofthe configuration of the tile 210, e.g., those settings for the tile 210which remain the same across different data path instructions. Forexample, the spoke RAM 320 is used to specify which input (e.g., one ofseveral sets of input communication lines 270A or input registers 350)of the tile 210 is the master synchronous input for each clock cycle, asthe selection control of input multiplexer(s) 355. This is significantbecause the instruction index to select an instruction (from theinstruction RAM 315) for a given time slice of the tile 210, and athread ID (TID), is provided on the master synchronous input. As aresult, even if the actual instruction index provided on an input 270Ato a given tile 210 may be varied (as described with reference to FIG.24 ), what set of inputs 270A that will have that selected instructionindex is not varied, so that any given tile 210 knows in advance whatinput it will use to receive a selected instruction index, independentlyof the instruction specified by that selected instruction index. Theconfigurations held in the spoke RAM 320 also usually designate whichoutputs 270B will be utilized for the selected instruction (or timeslice). The spoke RAM 320 read address input, i.e., the spoke index,comes from a counter that counts (modulo) from zero to the spoke countminus one. All tiles 210 within an HTF circuit cluster 205 generallyshould have the same spoke RAM input value each clock to have propersynchronous domain operation. The spoke RAM 320 also stores instructionindices and is also utilized to select instructions from the instructionRAM 315, so that a series of instructions may be selected for executionby the tile 210 as the count of the spoke RAM 320 changes, for a basetile 210 of a synchronous domain. For subsequent tiles in thesynchronous domain, the instruction index may be provided by a previoustile 210 of the synchronous domain. This aspect of the spoke RAM 320 isalso discussed with reference to FIG. 24 , as the spoke RAM 320 ishighly inventive, enabling dynamic self-configuration andreconfiguration of a HTF circuit cluster 205.

The spoke RAM 320 also specifies when a synchronous input 270A is to bewritten to tile memory 325. This situation occurs if multiple inputs arerequired for a tile instruction, and one of the inputs arrives early.The early arriving input can be written to tile memory 325 and thenlater read from the memory 325 when the other inputs have arrived. Thetile memory 325, for this situation, is accessed as a FIFO. The FIFOread and write pointers are stored in the tile memory region ram.

Each tile 210 contains one or more memories 325, and typically each arethe width of the data path (64-bits), and the depth will be in the rangeof 512 to 1024 elements, for example. The tile memories 325 are used tostore data required to support data path operations. The stored data canbe constants loaded as part of a kernel's cluster 205 configuration, orvariables calculated as part of the data flow. The tile memory 325 canbe written from the synchronous mesh communication network 275 as eithera data transfer from another synchronous domain, or the result of a loadoperation initiated by another synchronous domain. The tile memory isonly read via synchronous data path instruction execution.

Tile memory 325 is typically partitioned into regions. A small tilememory region RAM stores information required for memory region access.Each region represents a different variable in a kernel. A region canstore a shared variable (i.e., a variable shared by all executingthreads). A scalar shared variable has an index value of zero. An arrayof shared variables has a variable index value. A region can store athread private variable indexed by the TID identifier. A variable can beused to transfer data from one synchronous domain to the next. For thiscase, the variable is written using the XID_WR identifier in the sourcesynchronous domain, and read using the XID_RD identifier in thedestination domain. Finally, a region can be used to temporarily storedata produced by a tile 210 earlier in the synchronous data path untilother tile data inputs are ready. For this case, the read and writeindices are FIFO pointers. The FIFO pointers are stored in the tilememory region RAM.

The tile memory region RAM typically contains the following fields:

1. A Region Index Upper, which are the upper bits of a tile memoryregion index. The lower index bits are obtained from an asynchronousfabric message, the TID, XID_WR or XID_RD identifiers, or from the FIFOread/write index values. The Region Index Upper bits are OR'ed with thelower index bits to produce the tile memory 325 index.

2. A Region SizeW, which is the width of a memory region's lower index.The memory region's size is 2^(SizeW) elements.

3. A Region FIFO Read Index, which is the read index for a memory regionacting as a FIFO.

4. A Region FIFO Write Index, which is the write index for a memoryregion acting as a FIFO. The tile performs compute operations for theHTF 200.

The compute operations are performed by configuring the data path withinthe tile 210. There are two functional blocks that perform all computefor the tile 210: the multiply and shift operation circuit (“MS Op”)305, and the Arithmetic, Logical and Bit Operation circuit (“ALB Op”)310. The MS Op 305 and ALB Op 310 are under the control of theinstructions from the instruction RAM 315, and can be configured toperform two pipelined operations such as a Multiply and Add, or Shiftand AND, for example and without limitation. (In a representativeembodiment, all devices that support the HTF 200 would have the completesupported instruction set. This would provide binary compatibilityacross all devices. However, it may be necessary to have a base set offunctionality and optional instruction set classes to meet die sizetradeoffs. This approach is similar to how the RISC-V instruction sethas a base and multiple optional instruction subsets.) As illustrated inFIG. 9 , the outputs of the MS Op 305 and ALB Op 310 may be provided toregisters 312, or directly to other components, such as outputmultiplexers 395, conditional logic circuitry 370, and/or write maskgenerator 375.

The various operations performed by the MS Op 305 include, for exampleand without limitation: integer and floating point multiply, shift, passeither input, signed and unsigned integer multiply, signed and unsignedshift right, signed and unsigned shift left, bit order reversal,permutations, any and all of these operations as floating pointoperations, and interconversions between integer and floating point,such as double precision floor operations or convert floating point tointeger. The various operations performed by the ALB Op 310 include, forexample and without limitation: signed and unsigned addition, absolutevalue, negate, logical NOT, add and negate, subtraction A-B, reversesubtraction B-A, signed and unsigned greater than, signed and unsignedgreater than or equal to, signed and unsigned less than, signed andunsigned less than or equal to, comparison (equal or not equal to),logical operations (AND, OR, XOR, NAND, NOR, NOT XOR, AND NOT, OR NOT,any and all of these operations as floating point operations, andinterconversions between integer and floating point, such as flooroperations or convert floating point to integer.

The inputs to the ALB Op 310 and the MS Op 305 are from either thesynchronous tile inputs 270A (held in registers 350), from the internaltile memories 325, or from a small constant value provided within theinstruction RAM 315. The following Table 1, showing tile 210 data pathinput sources, lists the typical inputs for the ALB Op 310 and the MS Op305.

TABLE 1 Source Name Source Description SYNC_U Synchronous meshcommunication network 275 up link SYNC_D Synchronous mesh communicationnetwork 275 down link SYNC_L Synchronous mesh communication network 275left link SYNC_R Synchronous mesh communication network 275 right linkTILE_OUT Output of ALB Op 310 within the tile 210. RDMEM0_T Memory 0read data. Memory 325 region is indexed using TID from the MasterSynchronous Interface. RDMEM0_X Memory 0 read data. Memory 325 region isindexed using XID from the Master Synchronous Interface. RDMEM0_C Memory0 read data. Memory 325 region is indexed using instruction ram constantvalue. RDMEM0_V Memory 0 read data. Memory 325 region is indexed usingvalue received from a synchronous input, as variable indexing. RDMEM0_FMemory 0 read data. Memory 325 region is read using FIFO ordering.RDMEM0_Z Memory 0 read data. Memory 325 region is indexed using thevalue zero. RDMEM1_T Memory 1 read data. Memory 325 region is indexedusing TID from the Master Synchronous Interface. RDMEM1_X Memory 1 readdata. Memory 325 region is indexed using XID from the Master SynchronousInterface. RDMEM1_C Memory 1 read data. Memory 325 region is indexedusing instruction ram constant value. RDMEM1_V Memory 1 read data.Memory 325 region is indexed using value received from a synchronousinput, as variable indexing. RDMEM1_F Memory 1 read data. Memory 325region is read using FIFO ordering. RDMEM1_Z Memory 1 read data. Memory325 region is indexed using the value zero. CONST The data path input isthe zero extended constant value within an instruction. ITER_IDX Thedata path input is the zero extended loop iteration value, described ingreater detail below. ITER_W The data path input is the zero extendedloop iterator width value. See the loop section for more information.

Each of the outputs 270B of a tile 210, as part of the communicationlines 270 of the synchronous mesh communication network 275, areindividually enabled allowing clock gating of the disabled outputs. Theoutput of the ALB Op 310 can be sent to multiple destinations, shown inTable 2.

TABLE 2 Destination Name Destination Description SYNC_U Synchronous meshcommunication network 275 up link SYNC_D Synchronous mesh communicationnetwork 275 down link SYNC_L Synchronous mesh communication network 275left link SYNC_R Synchronous mesh communication network 275 right linkWRMEM0_Z Write Memory 0. Memory 325 region is written using the valuezero as the index. WRMEM0_C Write Memory 0. Memory 325 region is writtenusing the instruction constant field as the index. WRMEM0_T Write Memory0. Memory 325 region is written using the TID value as the index.

At a high level, and all as representative examples without limitation,the general operation of a tile 210 is as follows. The synchronous meshcommunication network 275 and synchronous domains of the various tiles210 are all scheduled as part of the program compilation and when theconfiguration is loaded into the system. Unless paused or stopped, atile 210 can execute its operations when all required inputs are ready,for example, data variables are in input registers 350 or memory 325,and are available to be read or taken from the registers or memory andused, as described in greater detail below. In a representativeembodiment, each pipeline stage may operate in a single clock cycle,while in other representative embodiments, additional clock cycles maybe utilized per pipeline stage. In a first pipeline stage 304, data isinput, such as into the AF input queues 360 and input registers 350, andoptionally directly into the memory 325. In a next pipeline stage 306,AF messages are decoded by AF state machine 345 and moved into memory325; the AF state machine 345 reads data from memory 325 or receivedfrom the output multiplexers 395 and generates a data packet fortransmission over the asynchronous packet network 265; data in the inputregisters 350 is moved into memory 325 or selected as operand data(using input multiplexers 355 and intermediate multiplexers 365), orpassed directly to output registers 380 for output on the synchronousmesh communication network 275; for example. In one or more of the nextpipeline stages 307 and 308, computations are performed by the ALB Op310 and/or the MS Op 305, write masks may be generated by write maskgenerator 375, and instructions (or instruction indices) may be selectedbased on test conditions in conditional (branch) logic circuitry 370. Ina next pipeline stage 309, outputs are selected using outputmultiplexers 395, and output messages (which may have been stored in theAF output queues 390) are transmitted on the asynchronous packet network265, and output data in any of the output registers 380 are transmittedon the synchronous mesh communication network 275.

FIG. 10 is a detailed block diagram of a representative embodiment of amemory control circuit 330 (with associated control registers 340) of ahybrid threading fabric configurable computing circuit (tile) 210. FIG.10 shows a diagram of the tile memory 325 read indexing logic of thememory control circuit 330, and is duplicated for each memory 325 (notseparately illustrated). The instruction RAM 315 has a field thatspecifies which region of the tile memory 325 is being accessed, and afield that specifies the access indexing mode. The memory region RAM 405(part of the control registers 340) specifies a region read mask thatprovides the upper memory address bits for the specific region. The maskis OR'ed (OR gate 408) in with the lower address bits supplied by theread index selection mux 403. The memory region RAM 405 also containsthe read index value when the tile memory 325 is accessed in FIFO mode.The read index value in the RAM 405 is incremented and written back whenaccessing in FIFO mode. The memory region RAM 405, in variousembodiments, may also maintain a top of TID stack through nested loops,described below.

FIG. 10 also shows the control information (INSTR, XID, TID) for thesynchronous mesh communication network 275 is required a few clocksearlier than the data input. For this reason, the control information issent out of the previous tile 210 a few clocks prior to sending thedata. This staging of synchronous mesh communication network 275information reduces the overall pipeline stages per tile 210, but itmakes it challenging to use a calculated value as an index to the tilememories 325. Specifically, the synchronous mesh communication network275 data may arrive too late to be used as an index into the tilememories 325. The architected solution to this problem is to provide thecalculated index from a previous tile 210 in a variable index registerof the control registers 340. Later, another input 270A causes thevariable index register to be used as a tile memory 325 index.

The asynchronous packet network 265 is used to perform operations thatoccur asynchronous to a synchronous domain. Each tile 210 contains aninterface to the asynchronous packet network 265 as shown in FIG. 9 .The inbound interface (from communication lines 280A) is the AF inputqueues 360 (as a FIFO) to provide storage for messages that cannot beimmediately processed. Similarly, the outbound interface (tocommunication lines 280B) is the AF output queues 390 (as a FIFO) toprovide storage for messages that cannot be immediately sent out. Themessages over the asynchronous packet network 265 can be classified aseither data messages or control. Data messages contain a 64-bit datavalue that is written to one of the tile memories 325. Control messagesare for controlling thread creation, freeing resources (TID or XID), orissuing external memory references. The following Table 3 lists theasynchronous packet network 265 outbound message operations:

TABLE 3 Operation Name Operation Description FREE_XID A message sent tothe base tile 210 of a synchronous domain to free an XID_RD. FREE_TID Amessage sent to the base tile 210 of a synchronous domain to free a TID.CONT_X A first type of continuation message sent to the base tile 210 ofa synchronous domain to indicate that a thread should be initiated aftera specified number of completion messages have been received. CONT_T Asecond type of continuation message sent to the base tile 210 of asynchronous domain to indicate that a thread should be initiated after aspecified number of completion messages have been received. INNER_LOOP Amessage sent to initiate an inner loop of a strip mined loop construct.The message specifies the number of loop iterations to perform. A workthread is initiated for each iteration. The iteration index is availablewithin the base tile 210 as an input to the data path source multiplexer365 (ITER_IDX). OUTER_LOOP A message sent to initiate an outer loop of astrip mined loop construct. The message specifies the number of loopiterations to perform. A work thread is initiated for each iteration.The iteration index is available within the base tile 210 as an input tothe data path source multiplexer 365 (ITER_ IDX). COMP A completionmessage is sent to indicate a synchronous domain work thread hascompleted. A base tile 210 counts the received completion messages inconjunction with receiving a call or continue message in order to allowa subsequent work thread to be initiated. The message sends the TIDidentifier as the pause table index, described below. CALL A callmessage is sent to continue a work thread on the same or anothersynchronous domain. A TID and/or an XID can optionally be allocated whenthe work thread is initiated. CALL_DATA A call data message is sent tocontinue a work thread on the same or another synchronous domain. A TIDand/or an XID can optionally be allocated when the work thread isinitiated. This message sends 128 bits (two 64- bit values) to bewritten to tile memory 325 within the base tile 210, along with a maskindicating which bytes of the 128 bit value to write. This is generallyalso the case for all asynchronous messages DATA_R A message is sent towrite to tile memory 325 of the destination tile 210. The TID value isused to specify the write index for the destination tile's memory. Acompletion messageis sent once the tile memory 325 is written tospecified base tile 210. DATA_X A message is sent to write to tilememory 325 of the destination tile 210. The XID_WR value is used tospecify the write index for the destination tile's memory 325. Acompletion message is sent once the tile memory 325 is written tospecified base tile 210. LD_ADDR_T A message is sent to the MemoryInterface 215 to specify the address for a memory load operation. TheTID identifier is used as the write index for the destination tile'smemory. LD_ADDR_X A message is sent to the Memory Interface 215 tospecify the address for a memory load operation. The XID_WR identifieris used as the write index for the destination tile's memory LD_ADDR_Z Amessage is sent to the Memory Interface 215 to specify the address for amemory load operation. Zero is used as the write index for thedestination tile's memory. ST_ADDR A message is sent to the MemoryInterface 215 to specify the address for a memory store operation.ST_DATA A message is sent to the Memory Interface 215 to specify thedata for a memory store operation.

The asynchronous packet network 265 allows messages to be sent andreceived from tiles 210 in different synchronous domains. There are fewsituations where it makes sense for a synchronous domain to send amessage to itself, such as when a synchronous domain's base tile 210allocates a TID, and the TID is to be freed by that same synchronousdomain.

The asynchronous packet network 265 can become congested if thesynchronous domains of the various tiles 210 generate and send messagesfaster than they can be received, routed and processed by theasynchronous packet network 265 or receiving endpoints. In thissituation, a backpressure mechanism is provided to alleviate any suchcongestion. FIG. 22 is a block diagram of a representative flow controlcircuit 385. Generally, there is at least one flow control circuit 385per HTF circuit cluster 205. The tile 210 asynchronous fabric outputqueues 390 will hold messages as they wait to be sent on theasynchronous packet network 265. A predetermined threshold is providedfor the output queue 390 that, when reached, will cause an output queue390 of a tile 210 to generate an indicator, such as setting a bit, whichis asserted as a “stop” signal 382 on a communication line 384 providedto the flow control circuit 385. Each communication line 384 from a tile210 in a HTF circuit cluster 205 is provided to the flow control circuit385. The flow control circuit 385 has one or more OR gates 386, whichwill continue to assert the stop signal 382 on communication line 388distributed to all tiles 210 within the affected HTF circuit cluster205, for as long as any one of the tiles 210 is generating a stop signal382.

The stop signal 382 may be distributed over a dedicated communicationline 388 which is not part of either the synchronous mesh communicationnetwork 275 or the asynchronous packet network 265 as illustrated, orover the synchronous mesh communication network 275. In a representativeembodiment, there is a single stop signal 382 that all tile outputqueues 390 within a HTF circuit cluster 205 can assert, and all tiles210 within that HTF circuit cluster 205 are held (paused or stopped)when a stop signal 382 is asserted. This stop signal 382 continues toallow all AF input queues 360 to receive AF messages and packets,avoiding deadlock, but also causes all synchronous domain pipelines tobe held or paused (which also prevents the generation of additional AFdata packets). The stop signal 382 allows the asynchronous packetnetwork 265 to drain the tile 210 output queues 390 to the point wherethe number of messages in the output queue 390 (of the triggering outputqueue(s) 390) has fallen below the threshold level. Once the size of theoutput queue 390 has fallen below the threshold level, then the signalover the communication line 384 is returned to zero (the stop signal 382is no longer generated) for that tile 210. When that has happened forall of the tiles 210 in the HTF circuit cluster 205, the signal oncommunication line 388 also returns to zero, meaning the stop signal isno longer asserted, and ending the stop or pause on the tiles 210.

The first or “base” tile 210 of a synchronous domain has theresponsibility to initiate threads of work through the multi-tile 210synchronous pipeline. A new thread can be initiated on a predeterminedcadence. The cadence interval referred to herein as the “spoke count”,as mentioned above. For example, if the spoke count is three, then a newthread of work can be initiated on the base tile 210 every three clocks.If starting a new thread is skipped (e.g., no thread is ready to start),then the full spoke count must be waited before another thread can bestarted. A spoke count greater than one allows each physical tile 210 tobe used multiple times within the synchronous pipeline. As an example,if a synchronous domain is executed on a single physical tile and thespoke count is one, then the synchronous domain can contain only asingle tile time slice. If, for this example the spoke count is four,then the synchronous domain can contain four tile time slices.Typically, a synchronous domain is executed by multiple tiles 210interconnected by the synchronous links of the synchronous meshcommunication network 275. A synchronous domain is not restricted to asubset of tiles 210 within a cluster 205, i.e., multiple synchronousdomains can share the tiles 210 of a cluster 205. A single tile 210 canparticipate in multiple synchronous domains, e.g., spoke 0, a tile 210works on synchronous domain “A”; spoke 1, that tile 210 works onsynchronous domain “B”; spoke 2, that tile 210 works on synchronousdomain “A”; and spoke 3, that tile 210 works on synchronous domain “C”.Thread control for a tile is described below with reference to FIG. 11 .

FIG. 11 is a detailed block diagram of a representative embodiment of athread control circuit 335 (with associated control registers 340) of ahybrid threading fabric configurable computing circuit (tile) 210.Referring to FIG. 11 , several registers are included within the controlregisters 340, namely, a TID pool register 410, an XID pool register415, a pause table 420, and a completion table 422. In variousembodiments, the data of the completion table 422 may be equivalentlyheld in the pause table 420, and vice-versa. The thread controlcircuitry 335 includes a continue queue 430, a reenter queue 445, athread control multiplexer 435, a run queue 440, an iteration increment447, an iteration index 460, and a loop iteration count 465.Alternatively, the continue queue 430 and the run queue 440 may beequivalently embodied in the control registers 340.

FIG. 12 is a diagram of tiles 210 forming first and second synchronousdomains 526, 538 and representative asynchronous packet networkmessaging. One difficulty with having an asynchronous packet network 265is that required data may arrive at tiles 210 at different times, whichcan make it difficult to ensure that a started thread can run tocompletion with a fixed pipeline delay. In representative embodiments,the tiles 210 forming a synchronous domain do not execute a computethread until all resources are ready, such as by having the requireddata available, any required variables, etc., all of which have beendistributed to the tiles over the asynchronous packet network 265, andtherefore may have arrived at the designated tile 210 at any of varioustimes. In addition, data may have to be read from system memory 125 andtransferred over the asynchronous packet network 265, and therefore alsomay have arrived at the designated tile 210 at any of various times.

To provide for a thread to run to completion with a fixed pipelinedelay, the representative embodiments provide a completion table 422 (orpause table 420) indexed by a thread's TID at the base tile 210 of asynchronous domain. The completion table 422 (or pause table 420)maintains a count of dependency completions that must be received priorto initiating execution of the thread. The completion table 422 (orpause table 420) includes a field named the “completion count”, which isinitialized to zero at reset. Two types of AF messages are used tomodify the count field. The first message type is a thread start orcontinue message, and increments the field by a count indicating thenumber of dependences that must be observed before a thread can bestarted in the synchronous domain. The second AF message type is acompletion message and decrements the count field by one indicating thata completion message was received. Once a thread start message isreceived, and the completion count field reaches zero, then the threadis ready to be started.

As illustrated in FIG. 12 , a tile 210B of a first synchronous domain526 has transmitted an AF memory load message (293) to the memoryinterface 215 over the asynchronous packet network 265, which in turnwill generate another message (296) to system memory 125 over the firstinterconnect network 150 to obtain the requested data (returned inmessage 297). That data, however, is to be utilized by and istransmitted (message 294) to a tile 201E in the second synchronousdomain 538. As the first synchronous domain 526 has completed itsportion of the pipeline, one of the tiles (210C) in the firstsynchronous domain 526 transmits an AF continue message (291) to thebase tile 210D of the second synchronous domain 538. That AF continuemessage (291) includes the TID of the thread of the first synchronousdomain 526 (e.g., TID=1, which was also included in the other messagesof FIG. 12 ), and a completion count of 1, indicating that the base tile210D of the second synchronous domain 538 will be waiting to start thethread on the second synchronous domain 538 until it has received onecompletion message.

Accordingly, when a tile 210 receives such data, such as tile 210E inFIG. 12 , it acknowledges that receipt by sending a completion message(with the thread ID (TID)) back to the base tile 210, here, base tile210D of the second synchronous domain 538. As part of the configurationprovided to the base tile 210 (at initial configuration or as part ofthe continue message), and stored in the completion table 422 (or pausetable 420) as a completion count, the base tile 210D knows how many suchcompletion messages the base tile 210 must receive in order to commenceexecution by the tiles 210 of the synchronous domain, in this case, thesecond synchronous domain 538. As completion messages are received bythe base tile 210, for the particular thread having that TID, thecompletion count of the pause table is decremented, and when it reacheszero for that thread, indicating all required completion messages havebeen received, the base tile 210 can commence execution of the thread.To commence execution, the TID of the thread is transferred to thecontinue queue 430, from which it is selected to run (at the appropriatespoke count for the appropriate time slice of the tile 210). It shouldbe noted that completion messages are not required for data which isdetermined during execution of the thread and which may be transferredbetween tiles 210 of the synchronous domain over the synchronous meshcommunication network 275.

There are several advantages to this type of thread control. This threadcontrol waits for all dependencies to be completed prior to starting thethread, allowing the started thread to have a fixed synchronousexecution time. The fixed execution time allows for the use of registerstages throughout the pipeline instead of FIFOs. In addition, while onethread of a tile 210 may be waiting to execute, other threads may beexecuting on that tile 210, providing for a much higher overallthroughput, and minimizing idle time and minimizing unused resources.

Similar control is provided when spanning synchronous domains, such asfor performance of multiple threads (e.g., for related compute threadsforming a compute fiber). For example, a first synchronous domain willinform the base tile 210 of the next synchronous domain, in acontinuation message transmitted over the asynchronous packet network265, how many completion messages it will need to receive in order forit to begin execution of the next thread. Also for example, foriterative looping spanning synchronous domains, a first synchronousdomain will inform the base tile 210 of the next synchronous domain, ina loop message (having a loop count and the same TID) transmitted overthe asynchronous packet network 265, how many completion messages itwill need to receive in order for it to begin execution of the nextthread.

It should also be mentioned that various delays may need to beimplemented, such as when first data is available from a first tile 210,while second data is still being determined by a second tile 210, bothof which will be needed in a next calculation by a third tile 210. Forsuch cases, delay may be introduced either at the output registers 380at the first tile 210 which created the first data, or in a tile memory325 of the third tile 210. This delay mechanism is also applicable todata which may be transferred from a first tile 210, using a second tile210 as a pass-through, to a third tile 210.

The pause table 420 is used to hold or pause the creation of a newsynchronous thread in the tile 210 until all required completionmessages have been received. A thread from a previous synchronous domainsends a message to a base tile 210 that contains the number ofcompletion messages to expect for the new synchronous thread, and theaction to take when all of the completion messages have been received.The actions include: call, continue, or loop. Many pause operations aretypically active concurrently. All messages for a specific pauseoperation (i.e., a set of pause and completion messages) will have thesame pause index within the respective messages. The pause index is theTID from the sending tile 210. Pause table 420 entries are initializedto be inactive with a completion delta count of zero. Receiving a pausemessage increments the delta count by the number of required completioncounts, and sets the pause table 420 entry to active. Receiving acompletion message decrements the delta count by one. It should be notedthat a completion message may arrive prior to the associated pausemessage, resulting in the delta count being negative. When a pause table420 entry is active with a delta count of zero then the associatedactivity (e.g., the new thread) is initiated (and the pause table 420entry is de-activated).

The continuation (or call) queue 430 holds threads ready to be startedon a synchronous domain. A thread is pushed into the continuation queue430 when all completions for a call operation are received. It should benoted that threads in the continuation queue 430 may require a TIDand/or XID to be allocated before the thread can be started on asynchronous domain, e.g., if all TIDs are in use, the threads in thecontinuation queue 430 can be started once a TID is freed and availablei.e., the thread may be waiting until TID and/or XIDs are available.

The reenter queue 445 holds threads ready to be started on a synchronousdomain, with execution of those threads having priority over those inthe continuation queue 430. A thread is pushed into the reenter queue445 when all completions for a continue operation are received, and thethread already has a TID. It should be noted that that threads in thereenter queue 445 cannot require allocation of a TID. Separate reenterand continue (or continuation) queues 445, 430 are provided to avoid adeadlock situation. A special type of continue operation is a loop. Aloop message contains a loop iteration count. The count is used tospecify how many times a thread is to be started once the pauseoperation completes.

An optional priority queue 425 may also be implemented, such that anythread having a thread identifier in the priority queue 425 is executedprior to execution of any thread having a thread identifier in thecontinuation queue 430 or in the reenter queue 445.

An iteration index 460 state is used when starting threads for a loopoperation. The iteration index 460 is initialized to zero andincremented for each thread start. The iteration index 460 is pushedinto the run queue 440 with the thread information from the continuequeue 430. The iteration index 460 is available as a selection to thedata path input multiplexer 365 within the first tile (base tile) 210 ofthe synchronous domain.

The loop iteration count 465 is received as part of a loop message,saved in the pause table 420, pushed into the continue queue 430, andthen used to determine when the appropriate number of threads have beenstarted for a loop operation.

The run queue 440 holds ready-to-run threads that have assigned TIDsand/or XIDs and can execute when the appropriate spoke count clocks haveoccurred. The TID pool 410 provides unique thread identifiers (TIDs) tothreads as they are started on the synchronous domain. Only threadswithin the continuation queue 430 can acquire a TID. The XID pool 415provides unique transfer identifiers (XIDs) to threads as they arestarted on the synchronous domain. Threads from the continue queue 430can acquire an XID. An allocated XID becomes the XID_WR for the startedthread.

For any given or selected program to be executed, the code orinstructions for that program, written or generated in any appropriateor selected programming language, are compiled for and loaded into thesystem 100, including instructions for the HTP 300 and HTF circuits 200,and any which may be applicable to the host processor 110, to providethe selected configuration to the system 100. As result, various sets ofinstructions for one or more selected computations are loaded into theinstruction RAMs 315 and the spoke RAMs 320 of each tile 210, and loadedinto any of the various registers maintained in the memory interfaces215 and dispatch interface 225 of each tile 210, providing theconfigurations for the HTF circuits 200, and depending upon the program,also loaded into the HTP 300.

A kernel is started with a work descriptor message that contains zero ormore arguments, typically generated by the host processor 110 or the HTP300, for performance by one or more HTF circuits 200, for example andwithout limitation. The arguments are sent within the work descriptor AFmessage to the dispatch interface 225. These arguments providethread-specific input values. A host processor 110 or HTP 300, using itsrespective operating system (“OS”) can send a “host” message to a kernelthat initializes a tile memory 325 location, with such host messagesproviding non-thread specific values. A typical example is a hostmessage that sends the base address for a data structure that is used byall kernel threads.

A host message that is sent to a kernel is sent to all HTF circuitclusters 205 where that kernel is loaded. Further, the order of sendinghost messages and sending kernel dispatches is maintained. Sending ahost message essentially idles that kernel prior to sending the message.Completion messages ensure that the tile memory 325 writes havecompleted prior to starting new synchronous threads.

The control messaging over the asynchronous packet network 265 is asfollows:

(1) The dispatch interface 225 receives the host message and sends an AFData message to the destination tile 210. The destination tile 210writes the selected memory with the data of the AF Data message.

(2) The destination tile 210 sends an AF Complete message to thedispatch interface 225 acknowledging that the tile write is complete.

(3) The dispatch interface 225 holds all new kernel thread starts untilall message writes have been acknowledged. Once acknowledged, thedispatch interface 225 transmits an AF Call message to the base tile ofthe synchronous domain to start a thread.

The dispatch interface 225 is responsible for managing the HTF circuitcluster 205, including: (1) interactions with system 100 software toprepare the HTF circuit cluster 205 for usage by a process; (2)dispatching work to the tiles 210 of the HTF circuit cluster 205,including loading the HTF circuit cluster 205 with one or more kernelconfigurations; (3) saving and restoring contexts of the HTF circuitcluster 205 to memory 125 for breakpoints and exceptions. As mentionedabove, the registers 475 of the dispatch interface 225 may include awide variety of tables to track what has been dispatched to and receivedfrom any of the various tiles 210, such as tracking any of the messagingutilized in representative embodiments. The dispatch interface 225primitive operations utilized to perform these operations are listed inTable 4.

TABLE 4 Primitive Operation Location Initiated by Operation DescriptionHTF Cluster Load HTF Application A HTF circuit cluster 205 checks eachKernel Dispatch received work descriptor to determine if theConfiguration Interface required kernel configuration matches thecurrently loaded configuration. If the work descriptor's kernelconfiguration does not match the currently loaded configuration, thenthe HTF circuit cluster 205 waits for all previous work to complete andloads the new kernel configuration. Each work descriptor has the virtualaddress for the required kernel configuration. HTF Cluster Reset HTF OSReset all state within a HTF circuit cluster Dispatch 205 to allow a newkernel configuration or Interface kernel context to be loaded. HTFCluster Store HTF OS An HTF circuit cluster 205 can pause ContextDispatch execution due to an exception or breakpoint. Interface The HTFcircuit cluster 205 sends an interrupt to the OS to inform it of theevent. The OS determines if process context must be stored to memory fordebugger access. If process context is required, then the OS initiatesthe operation by interacting with the dispatch interface 225 of the HTFcircuit cluster 205. HTF Cluster Load HTF OS The context for an HTFcircuit cluster 205 Context Dispatch can be loaded from memory inpreparation to Interface resume execution. The OS initiates theoperation by interacting with the dispatch interface 225 of the HTFcircuit cluster 205. HTF Cluster Pause HTF OS The OS may need to pauseexecution on Dispatch running HTF circuit clusters 205 when theInterface owning process needs to be stopped. The process may need to bestopped if an exception or breakpoint occurred by a processor ordifferent HTF circuit cluster 205, or the process received a LinuxSignal. The OS initiates the pause by interacting with the dispatchinterface 225. HTF Cluster HTF OS Execution of a paused HTF circuitcluster Resume Dispatch 205 can be resumed by removal of the pauseInterface signal of the HTF circuit cluster 205. The OS initiates theresume by interacting with the dispatch interface 225 of the HTF circuitcluster 205. HTF Cluster Is Idle HTF OS The OS may need to determinewhen an HTF Dispatch circuit cluster 205 is idle and ready to acceptInterface a new operation. The dispatch interface 225 has a number ofstate machines that perform various commands. These commands includecontext load, context store, pause, and configuration load. The OS mustensure that an HTF circuit cluster 205 is idle prior to issuing acommand.

FIGS. 16 and 17 provide an example of messaging and thread controlwithin a system 100, with an example computation provided to show howthe synchronous mesh communication network 275 and asynchronous packetnetwork 265 work together to execute a simple kernel, here, solving thesimple expression R=*A+B. To show such messaging, the computation hasbeen divided across two different synchronous domains 526 and 538. Thevariable B is passed as a host message to all HTF circuit clusters 205,and the address of A is passed as an argument to the call in the workdescriptor packet. The result R is passed back in the return data packetover the first interconnection network 150. The example does almost nocompute so the number of messages per compute performed is very high.The HTF circuits 200 have much higher performance when significantcomputation is performed within a loop such that the number of messagesper compute is low.

FIG. 16 is a diagram of representative hybrid threading fabricconfigurable computing circuits (tiles) 210 forming synchronous domainsand representative asynchronous packet network messaging for performanceof a computation by a HTF circuit cluster 205. FIG. 17 is a flow chartof representative asynchronous packet network messaging and execution byhybrid threading fabric configurable computing circuits (tiles) forperformance of the computation of FIG. 16 by a HTF circuit cluster 205.To begin, the host processor 110 sends a message (504) to the all HTFcircuit clusters 205 within the node, step 506. The message is the valueof the variable B. The message is contained in a single data packet,typically referred to as a work descriptor packet, that is written to adispatch queue 105 of the HIF 115 (illustrated in FIGS. 1 and 2 )associated with the process. The HIF 115 reads the message from thedispatch queue 105 and sends a copy of the packet to each HTF circuitcluster 205 assigned to the process. The dispatch interface 225 of theassigned HTF circuit cluster 205 receives the packet. It should also benoted that the HIF 115 performs various load balancing functions acrossall HTP 300 and HTF 200 resources.

The host processor 110 sends a call message (508) to one HTF circuitcluster 205 assigned to the process, step 510. The host processor 110can either manually target a specific HTF circuit cluster 205 to executethe kernel, or allow the HTF circuit cluster 205 to be automaticallyselected. The host processor 110 writes the call parameters to thedispatch queue associated with the process. The call parameters includethe kernel address, starting instruction, and the single argument(address of variable A). The host interface (HIF) 115 reads the queuedmessage and forwards the message as data packet on the firstinterconnection network 150 to the assigned HTF circuit cluster 205,typically the HTF circuit cluster 205 with the least load.

The dispatch interface 225 receives the host message (value of variableB), waits until all previous calls to the HTF circuit cluster 205 havecompleted, and sends the value to a first selected, destination tile210H using an AF message (512) over the asynchronous packet network 265,step 514. The dispatch interface 225 has a table of information, storedin registers 475, for each possible host message that indicates thedestination tile 210H, tile memory 325 and memory region (in RAM 405)for that tile 210H. The tile 210H uses the message information to writethe value to a memory 325 in the tile 210H, and once the value iswritten to tile memory, then a write completion AF message (516) is sentvia the asynchronous packet network 265 back to the dispatch interface225, step 518.

The dispatch interface 225 waits for all message completion messages toarrive (in this case just a single message). Once all completionmessages have arrived, then the dispatch interface 225 sends the callargument (address of variable A) in an AF message (520) to a secondselected destination tile 210B for the value to be written into tilememory 325, step 522. The dispatch interface 225 has a call argumentstable stored in registers 475 that indicates the destination tile 210B,tile memory 325 and memory region (in RAM 405) for that tile 210B.

The dispatch interface 225 next sends an AF call message (524) to thebase tile 210A of the first synchronous domain 526, step 528. The AFcall message indicates that a single completion message should bereceived before the call can start execution through the synchronoustile 210 pipeline. The required completion message has not arrived sothe call is paused.

Once the value is written to the tile memory 325 for that tile 210B,then a write completion message (530) is sent by the tile 210B via theasynchronous packet network 265 to the base tile 210A of the firstsynchronous domain 526, step 532.

The base tile 210A has received both the call message (524) and therequired completion message (530), and is now ready to initiateexecution on the synchronous domain 526 (tile pipeline). The base tile210A initiates execution by providing the initial instruction and avalid signal (534) to the tile 210B, via the synchronous meshcommunication network 275, step 536. The base tile 210A allocates an XIDvalue from an XID pool 415 for use in the first synchronous domain 526.If the XID pool 415 is empty, then the base tile 210A must wait to startthe synchronous pipeline until an XID is available.

As execution proceeds, the tile 210B or another tile 210E within thefirst synchronous domain 526 sends an AF continue message (540) to thebase tile 210C of a second synchronous domain 538, step 542. Thecontinue message contains the number of required completion messagesthat must arrive before the second synchronous domain 538 can initiateexecution (in this case a single completion message). The continuemessage also includes the transfer ID (XID). The XID is used as a writeindex in one synchronous domain (526), and then as a read index in thenext synchronous domain (538). The XID provides a common tile memoryindex from one synchronous domain to the next.

The tile 210B or another tile 210F within the first synchronous domain526 sends an AF memory load message (544) to the memory interface 215 ofthe HTF circuit cluster 205, step 546. The message contains a requestID, a virtual address, and the XID to be used as the index for writingthe load data to a destination tile (210G) memory 325.

The memory interface 215 receives the AF load message and translates thevirtual address to a node local physical address or a remote virtualaddress. The memory interface 215 uses the AF message's request ID toindex into a request table stored in registers 485 containing parametersfor the memory request. The memory interface 215 issues a load memoryrequest packet (548) for the first interconnection network 150 with thetranslated address and size information from the request table, step550.

The memory interface 215 subsequently receives a memory response packet(552) over the first interconnection network 150 with the load data(value for variable A), step 554. The memory interface 215 sends an AFmessage (556) to a tile 210G within the second synchronous domain 538,step 558. The AF message contains the value for variable A and the valueis written to tile memory using a parameter from the request tablestored in registers 485.

Once the value is written to tile memory, then an AF write completionmessage (560) is sent via the asynchronous packet network 265 to thebase tile 210C of the second synchronous domain 538, step 562.

The base tile 210C of the second synchronous domain 538 receives boththe continue message (540) and the required completion message (560) andis ready to initiate execution on the second synchronous domain 538(tile pipeline). The base tile 210C initiates execution by providing theinitial instruction and a valid signal (564) to a tile 210 of the secondsynchronous domain 538, step 566, such as tile 210H. The base tile 210Calso allocates an XID value from an XID pool for use in the secondsynchronous domain 538.

A tile 210H within the second synchronous domain performs the addoperation of the B value passed in from a host message and the A valueread from system memory 125, step 568. The resulting value is the Rvalue of the expression.

A tile 210J within the second synchronous domain sends an AF message(570) containing the R value to the dispatch interface 225, step 572.The AF message contains the allocated XID value from the base tile 210A.The XID value is used as an index within the dispatch interface 225 fora table stored in registers 475 that hold return parameters until thevalues have been read and a return message generated for transmissionover the first interconnection network 150.

An AF message (574) from the second synchronous domain (tile 210K) sendsthe XID value allocated in the first synchronous domain back to the basetile 210A to be returned to the XID pool, step 576. A firstinterconnection network 150 message (578) from the dispatch interface225 is sent to the HIF 115, step 580. The HIF writes the return workdescriptor to the dispatch return queue. Once the first interconnectionnetwork 150 has sent the return packet, the XID value is sent in an AFmessage (582) by the dispatch interface 225 to the base tile 210C of thesecond synchronous domain 538 to be returned to the XID pool, step 584.

It should be noted that in this example of FIGS. 16 and 17 , many tiles210 have been utilized in order to show a wide variety of AF messageswhich may be utilized for thread control. In practice, especially forsuch a simple computation, appreciably fewer tiles 210 can be utilized,such as to perform the computation completely within a singlesynchronous domain.

Another messaging example is provided for thread control across multiplesynchronous domains in FIGS. 18 and 19 , also using AF complete andcontinue messages on the asynchronous packet network 265. FIG. 18 is adiagram of representative hybrid threading fabric configurable computingcircuits (tiles) forming synchronous domains and representativeasynchronous packet network messaging for performance of a computationby a hybrid threading fabric circuit cluster. FIG. 19 is a flow chart ofrepresentative asynchronous packet network messaging and execution byhybrid threading fabric configurable computing circuits (tiles) forperformance of the computation of FIG. 18 by a hybrid threading fabriccircuit cluster.

For this example, the dispatch interface 225 sends a message to the basetile 210A of the first synchronous domain 526. The message starts athread on the first synchronous domain 526. The thread sends a threadcontinue message to a second synchronous domain 538. The continuemessage indicates that a thread is to be started on the secondsynchronous domain 538 when the specified number of completion messageshave been received. The first synchronous domain 526 sends a completionmessage to the second synchronous domain 538 causing the pause tocomplete and start the synchronous, second thread. The second threadsends a complete message back to the dispatch interface 225 indicatingthat the second synchronous thread completed, completing the dispatchedkernel. Additional messages are shown in FIG. 18 that free TID and XIDidentifiers.

The dispatch interface 225 has received a work descriptor packet (602),has ensured that the correct kernel configuration is loaded, hasdetermined that the XID and TID pools are non-empty, obtaining the XIDand TID values for a new work thread from TID and XID pools stored inregisters 475 within the dispatch interface 225, step 604. The dispatchinterface 225 starts kernel execution by sending an AF Call message(606) (with the allocated XID and TID values, e.g., XID=3 and (firsttype) TID=11) to the base tile 210A of a first synchronous domain 526,step 608. The base tile 210A receives the AF Call message (606),determines that the TID and XID pools (410, 415) are non-empty,allocates TID and XID values (e.g., XID_WR=7 and (second type) TID=1),and that the Spoke Ram 320 is selecting the Base Tile as the input forthe tile data path, step 610, so that it begins execution with a firstdesignated instruction designated by the instruction index held in itsSpoke Ram 320, rather than potentially executing an instruction from aninstruction index which may have been provided by a previous tile 210(e.g., as discussed in greater detail below regarding conditionalexecution).

The base tile 210A starts a first thread (612) through the firstsynchronous domain 526 with the assigned TID value (e.g., (second type)TID=1), XID_RD assigned the value from the AF Call message (606) (e.g.,XID_RD=3), XID_WR assigned the value obtained from the XID pool (e.g.,XID_WR=7), and TID from the AF Call message (606) (e.g., (first type)TID=11), step 614.

As the computation proceeds in the first synchronous domain 526, anothertile 210B within the first synchronous domain 526 sends an AF Continuemessage (616) to the base tile 210D of the second synchronous domain538, step 618. The AF Continue message (616) provides the informationnecessary to start a second thread on the second synchronous domain 538when the appropriate number of completion messages have arrived. The AFContinue message (616) includes a completion count field having a valuethat specifies the number of required completion messages. One of thetiles (210C) in the first synchronous domain 526 also transmits a freeXID (e.g., XID=3) message (641) to the dispatch interface 225.

The AF Continue message (616) can include either the TID or XID_WR valueas the index into the pause table 420 on the destination base tile 210D.The pause table accumulates the received completion messages anddetermines when the requisite number have arrived and a new thread canbe started, step 620. The tile 210B that sends the AF Continue message(616) sends the selected TID or XID_WR value as the PID (pause tableindex) and changes the synchronous domain's downstream TID value to theselected value (e.g., (first type) TID=11, (second type) TID=1). Thisnew TID value is passed in all AF completion messages to be used as theindex into the pause table 420 of the base tile 210D.

An AF Complete message (622) is sent to the base tile 210D of the secondsynchronous domain 538 with the TID value (e.g., (first type) TID=11),step 624. The AF Complete message (622) decrements the entry of a deltacount field in the pause table 420 of the base tile 210D. The AFComplete message (622) and the AF Continue message (616) could arrive inany order. The last message to arrive will observe that the AF Continuemessage (616) has arrived and the delta count field in the pause table420 has reached zero. This condition indicates that the pause hascompleted and a second synchronous thread (626) can be started. The basetile 210D also determines or observes that the pause operation hascompleted, determines that the XID identifier pool is non-empty andallocates an XID (e.g., XID=5), and that the Spoke Ram is selecting theBase Tile as the input for the tile data path, step 628.

The base tile 210D then starts the second synchronous thread (626)through the second synchronous domain 538, step 630, with TID and XID_RDassigned the value obtained from the AF Continue message (616) (e.g.,(first type) TID=11, (second type) TID=1, XID_RD=7). The XID_WR isassigned the value obtained from the XID pool in step 628 (e.g.,XID_WR=5).

When the computations of the second synchronous thread (626) havecompleted, several housekeeping messages are sent by the various tiles210 of the second synchronous domain 538. An AF Free TID message (632)is sent to the base tile 210A of the first synchronous domain 526, step634, and the receiving base tile 210A adds the TID value to the TID pool410, step 636, so it is available once again for use. An AF Free XIDmessage (638) is sent to the base tile 210A of the first synchronousdomain 526, step 640, and the receiving base tile 210 adds the XID valueto the XID pool 415, step 642, also so it is available once again foruse. An AF Complete message (644) is sent to the dispatch interface 225indicating that the second synchronous thread 626 has completed, step646. The dispatch interface 225 has a count of expected completionmessages. The AF Complete message (644) includes the XID_WR value andTID value ((first type) TID=11) of the second synchronous domain 538 tothe dispatch interface. The dispatch interface 225 then sends an AF FreeXID message (648) to the base tile 210D of the second synchronous domain538, step 650. The receiving base tile 210D then adds the XID value tothe XID pool 415, step 652, so it is available once again for use.

A data transfer operation is used to transfer data from one synchronousdomain to the next. Typically, a data transfer is used in conjunctionwith a load operation obtaining data from memory 125. Calculated datafrom the first synchronous domain 526 is needed in the secondsynchronous domain 538 once the load data has arrived at the secondsynchronous domain 538. In this case, a single pause is sent from thefirst synchronous domain 526 to the second synchronous domain 538 thatcontains the total count of completion messages from all load and datatransfer operations.

The data transfer operation between synchronous domains then utilizes avariation of step 624. Instead of sending an AF Complete message (622)in step 624, the first synchronous domain 526 sends an AF Data messageto the second synchronous domain 538 with data. The destination tile 210in the second synchronous domain 538 writes the data within the AF Datamessage to the selected tile memory 325. The tile 210 that receives theAF Data message then sends an AF Complete message to the base tile 210of the second synchronous domain 538. The base tile 210 of the secondsynchronous domain 538 may then launch the second thread on the secondsynchronous domain 538 once the load data has arrived at the secondsynchronous domain 538.

Control over iterative thread looping across synchronous domainsutilizes a similar control messaging schema. The loop message flowallows multiple synchronous domain starts from a single loop message.Each of the started synchronous threads is able to access its iterationindex. FIG. 20 is a diagram of representative hybrid threading fabricconfigurable computing circuits (tiles) forming synchronous domains andrepresentative asynchronous packet network messaging for performance ofa loop in a computation by a hybrid threading fabric circuit cluster.FIG. 21 is a flow chart of representative asynchronous packet networkmessaging and execution by hybrid threading fabric configurablecomputing circuits (tiles) for performance of the loop in a computationof FIG. 20 by a hybrid threading fabric circuit cluster.

FIG. 20 shows three synchronous domains, first synchronous domain 526,second synchronous domain 538, and third synchronous domain 654. Thefirst synchronous domain 526 is used for pre-loop setup, the secondsynchronous domain 538 is started with an iteration count (IterCnt) forthe number of threads, and the final, third synchronous domain 654 ispost-loop. It should be noted that loops can be nested, as well, usingadditional layers of indexing, discussed in greater detail below.

Referring again to FIG. 11 , the control registers 340 include acompletion table 422 (or pause table 420). For loops, two kinds ofcompletion information are maintained in the completion table 422, afirst completion count pertaining to the number of completion messageswhich should arrive before a thread may start, as discussed above, and asecond, loop or iteration (completion) count, to track how many loopthreads have been started and completed. A loop is started by sending anAF loop message containing a loop count (and various TIDs, discussedbelow) to the base tile 210 of a synchronous domain. The loop count isstored in the completion table 422 (or pause table 420), and is used todetermine the number of times a new thread is started on the synchronousdomain. In one embodiment, each thread is started with a new TIDobtained from the TID pool 410. Each active thread has a unique TIDallowing thread private variables, for example. The threads of nestedloops are provided with access to the data or variables of its own TID,plus the TIDs of the outer loops. In a second embodiment discussedbelow, TIDs are re-used by successive threads of the loop.

TIDs are returned to the TID pool 410 by an AF message being sent from atile within a synchronous domain when the thread is terminating, whichmay be either an AF Complete message, or for the second embodiment, anAF reenter message. This can also be accomplished by a Free TID messageto the base tile 210. The AF message that returns the TID to the pool orre-uses the TID also is used by the loop base tile 210 to maintain acount of the number of active loop threads in the loop count of thecompletion table 422 (or pause table 420). When the number of activeloop threads reaches zero, then the loop is complete. When the loopcompletion is detected by the loop count going to zero, then an AFComplete message is sent to the post-loop synchronous domain informingof the completion. This mechanism provides for minimal (if not zero)idle cycles for nested loops, resulting in better performance.

Referring to FIGS. 20 and 21 , the first synchronous domain 526(illustrated as tile 210B, although it can be from any other tile in thefirst synchronous domain 526) sends an AF Continue message (656) to thebase tile 210D of the third, post-loop synchronous domain 654, step 658,to wait for the loop completion message (which will be from the secondsynchronous domain 538). One of the tiles in the first synchronousdomain 526, illustrated as tile 210B, sends an AF Loop message (660)with the iteration (loop) count to the base tile 210C of the loopdomain, which is the second synchronous domain 538, step 664. The basetile 210C starts the loop (IterCnt) threads (662, e.g., 662 ₀, 662 ₁,through 662 _(N-1), where “N” is the iteration count (IterCnt)) on thesecond synchronous domain 538. Each thread 662 has the same TID andXID_RD identifiers. The XID_WR identifier is allocated by the loop basetile 210C if enabled. The iteration index (i.e., ordered from zero toIterCnt-1 (N-1)) is accessible as a data path multiplexer selection inthe base tile 210C of the loop domain.

Each iteration of the loop domain then sends an AF Complete message(666) back to the base tile 210C of the second synchronous (loop) domain538, step 668. It should be noted that the second synchronous domain 538shown in FIG. 20 may actually be several synchronous domains. For thecase in which multiple synchronous domains form the loop, the threads ofthe last synchronous domain of the loop should transmit the AF Completemessages (666), so that the post-loop third synchronous domain 654properly waits for all loop operations to complete. Once the base tile210C of the second synchronous (loop) domain 538 has received alliteration AF Complete messages (666), it then sends a loop AF Completemessage (or an AF continuation message) (670) to the base tile 210D ofthe third (post-loop) synchronous domain 654.

For looping, including nested and doubly-nested looping, severaladditional and novel features are utilized, such as to minimize idletime, including a reenter queue 445 and additional sub-TIDs, such as aTID₂ for the outermost loop, a TID₁ for the middle or intermediate loop,and a TID₀ for the innermost loop, for example and without limitation.Each thread that is executing in the loop than also has a unique TID,such as TID₂s 0-49 for an outer loop which will have fifty iterations,which are also utilized in the corresponding completion messages wheneach iteration completes execution, also for example and withoutlimitation.

Referring again to FIG. 11 , several novel mechanisms are provided tosupport efficient looping, and to minimize idle time. For example, loopswith data-dependent end conditions (e.g., “while” loops) require thatthe end condition to be calculated as the loop is executed. Also forcontrol and execution of looping, a potential deadlock issue may ariseif all TIDs are allocated from the TID pool 410, but the thread at thehead of a queue for execution is a new loop, which cannot executebecause of a lack of available TIDs, blocking other looping threadswhich cannot complete and free up their assigned TIDs. Accordingly, inrepresentative embodiments, control registers 340 include two separatequeues for ready-to-run threads, with a first queue for initiating newloops (the continuation queue 430, also utilized for non-loopingthreads), and a second, separate queue (the reenter queue 445) for loopcontinuation. The continuation queue 430 allocates a TID from the TIDpool 410 to start a thread, as previously discussed. The reenter queue445 uses the previously allocated TID, as each iteration of a loopthread executes and transmits an AF reenter message with the previouslyallocated TID. Any thread (TID) in the reenter queue 445 will be movedinto the run queue 440 ahead of the threads (TIDs) which may be in theother queues (continuation queue 430). As a result, once a loop islaunched, loop iteration occurs very rapidly, with each next thread ofthe loop being launched rapidly through use of the separate reenterqueue 445, and further, without the potential for deadlock issues. Inaddition, the reenter queue 445 allows this rapid execution that is verysignificant for loops with data-dependent end conditions, which can nowrun effectively without interruption to the last iteration whichproduces the data-dependent end conditions.

Referring again to FIGS. 9 and 10 , control registers 340 include amemory region RAM 405. The memory region RAM 405, in variousembodiments, may also maintain a top of TID stack (with identifiers)through nested loops, described below. As mentioned above, each nestedloop initiates threads with a new (or re-used) set of TIDs. Threads of aloop may need to have access to its TID plus the TIDs of the outer loopthreads. Having access to the TIDs of each nested loop thread allowsaccess to each thread's private variables, such as the different levelor types of TIDs described above, TID₀, TID₁ and TID₂. The top of stackTID identifier indicates the TID for the active thread. The top of stackTID identifier is used to select which of the three TIDs (TID₀, TID₁ andTID₂) is used for various operations. These three TIDs and the top ofstack TID identifier are included in synchronous fabric controlinformation (or messages) transmitted on the synchronous meshcommunication network 275, so are known to each thread. Because multipleTIDs are included within a synchronous fabric message and include a topof stack TID identifier, the multiple TIDs allow a thread in a nestedloop to access variables from any level within the nested loop threads.The selected TID plus a tile memory region RAM 405 identifier is used toaccess a private thread variable.

Another novel feature of the present disclosure is the mechanism toorder loop thread execution to handle loop iteration dependencies, whichalso accommodates any delays in completion messages and data receivedover the asynchronous packet network 265. FIG. 23 is a diagram of tiles210 forming synchronous domains and representative asynchronous packetnetwork messaging and synchronous messaging for performance of a loop ina computation by a hybrid threading fabric circuit cluster. Asillustrated in FIG. 23 , multiple synchronous domains 682, 684, and 686,are involved in performance of a loop computation, a second synchronousdomain 682, a third synchronous domain 684, a fourth synchronous domain686, in addition to the pre-loop first synchronous domain 526 andpost-loop (fifth) synchronous domain 654. The loop computation may beany kind of loop, including nested loops, and in this case, there aredata dependencies within the various loops. For example, these datadependencies may occur within a single iteration, such as wheninformation is needed from memory 125, involving AF messaging over theasynchronous packet network 265. As a result, thread execution shouldproceed in a defined order, and not merely whenever any particularthread has a completion count of zero (meaning that thread is notwaiting on any data, with all completion messages for that thread havingarrived).

To provide ordered loop thread execution, in representative embodiments,additional messaging and additional fields are utilized in thecompletion table 422, for each loop iteration. The loop base tile 210Bprovides four pieces of information (for each loop iteration) that ispassed in synchronous messages 688 through each synchronous domain 682,684, 686 through the synchronous mesh communication network 275 (i.e.,passed to every successive tile 210 in that given synchronous domain),and AF continue messages 692 to the base tiles 210 of successivesynchronous domains via the asynchronous packet network 265 (which isthen passed in synchronous messages to each successive tile 210 in thatgiven synchronous domain). Those four fields of information are thenstored and indexed in the completion table 422 and utilized forcomparisons as the loop execution progresses. The four pieces ofinformation are: a first flag indicating the first thread of a set ofthreads for a loop, a second flag indicating the last thread of a set ofthreads for a loop, the TID for the current thread, and the TID for thenext thread. The TID for the current thread is obtained from a pool ofTIDs, and the TID for the next thread is the TID from the pool that willbe provided for the next thread. These four pieces of information areused by the base tile of each successive synchronous domain to orderthread starts. A thread can be started if its dependency count hasreached zero and the thread is the first thread for a loop, or thethread TID equals the next TID for the previously started thread.

Stated another way, the thread control circuitry 330 (which generallyincludes various state machines) checks the completion table 422, foreach thread which has received all data completions (so would otherwisebe ready-to-run), whether that thread is the next thread to run (havingthe next thread ID, e.g., TID=4), and if so, moves that thread (TID=4)into the run queue 440, and if not, does not start that thread (e.g., athread whose data completion count went to zero but has a TID=5) butmaintains the index of which TID is next to start. When the datacompletion for the thread having the next TID decrements to zero (TID=4in this case), so all completion messages have arrived, that thread isqueued for execution, and that thread (TID=4) which will be executinghas a next TID as well, in this case, its next TID is TID=5.Accordingly, when the thread having TID=4 has completed, the threadcontrol circuitry 330 checks the completion table 422 and now determinesthat the thread (TID=5) is the next thread ID, and queues that threadfor execution. When the thread ID is then the last TID, following itsexecution, an AF completion message (656) can be transmitted to thepost-loop base tile (in this case, 210E). It should be noted that thisuse of the additional fields in the completion table 422 may be extendedto any situation in which a particular ordering of thread executionshould be maintained.

FIG. 24 is a block and circuit diagram of a representative embodiment ofconditional branching circuitry 370. A synchronous domain, such as thefirst, second and third synchronous domains mentioned above, is a set ofinterconnected tiles, connected in a sequence or series through thesynchronous mesh communication network 275. Execution of a thread beginsat the first tile 210 of the synchronous domain, referred to as a basetile 210, and progresses from there via the configured connections ofthe synchronous mesh communication network 275 to the other tiles 210 ofthe synchronous domain. As illustrated in FIG. 24 , when a tile 210 hasbeen configured as a base tile 210 of the synchronous domain (as thoseconfigurations have been loaded into the HTF circuit 200 in advance ofrun time), the selection 374 of a configuration memory multiplexer 372is set equal to 1, which thereby selects the spoke RAM 320 to providethe instruction index for selection of instructions from the instructionRAM 315. For all other tiles 210 of the synchronous domain, theselection 374 of a configuration memory multiplexer 372 is set equal to0, which thereby selects an instruction index provided by the previoustile 210 in the sequence of tiles 210 of the synchronous domain. As aresult, the base tile 210 provides the instruction index (or theinstruction) to be executed to the next, second tile of the domain, viadesignated fields (or portions) of the communication lines (or wires)270B and 270A (which have been designated the master synchronous inputs,as mentioned above). By default, thereafter, this next tile 210, andeach succeeding tile 210 of the synchronous domain, will provide thesame instruction to each next tile 210 of the connected tiles 210 forexecution, as a static configuration.

In representative embodiments, however, a mechanism is provided fordynamic self-configuration, using the spoke RAM 320, the instruction RAM315, and the conditional branching circuitry 370. Referring to FIG. 24 ,for a current tile 210, the ALB Op 310 may be configured to generate anoutput which is the outcome of a test condition, such as whether oneinput is greater than a second input, for example. That test conditionoutput is provided to the conditional branching circuitry 370, oncommunication lines (or wires) 378. When the conditional branchingcircuitry 370 is enabled (through one or more bits of an instructionprovided on lines (or wires) 379), the test condition output is utilizedto select the next instruction index (or instruction) which is providedto the next tile 210 of the synchronous domain, such as to selectbetween “X” instruction or “Y” instruction for the next tile 210,providing conditional branching of the data path when the first or thesecond instruction is selected. Such conditional branching may also becascaded, such as when the next tile 210 is also enabled to provideconditional branching. By selecting the next instruction for one or moreof the next tiles 210, dynamic self-configuration andself-reconfiguration is enabled in each such HTF circuit cluster 205.

In a representative embodiment, the conditional branching circuitry 370has been arranged to select or toggle between two differentinstructions, depending on the test condition result. The branch enableis provided in one of the fields of the current (or currently next)instruction, and is provided to an AND gate 362 of the conditionalbranching circuitry 370, where it is ANDed with the test conditionoutput. Depending on the test condition output being a logical “0” or“1”, AND gate 362 will generate a logical “0” or “1” as an output, whichis provided as an input to OR gate 364. Another designated bit of aselected field of the currently next instruction index, typically theleast significant bit (“LSB”) of the next instruction index, is alsoprovided to the OR gate 364, where it is ORed with the output of the ANDgate 362. If the LSB of the next instruction index is a zero, and it isORed with a logical “1” of the output of the AND gate 362, then the nextinstruction index which is output has been incremented by one, providinga different next instruction index to the next tile 210. If the LSB ofthe next instruction index is a zero, and it is ORed with a logical “0”of the output of the AND gate 362, then the next instruction index whichis output has not been incremented by one, providing the same nextinstruction index to the next tile 210. As a result, the current tile210 has conditionally specified an alternate instruction for connectedtiles 210 to execute, enabling the performance of one or more casestatements in the HTF circuit cluster 205. The alternate instruction ischosen by having the current tile's data path produce a Booleanconditional value, and using the Boolean value to choose between thecurrent tile's instruction and the alternate instruction provided as thenext instruction index to the next tile 210 in the synchronous domain.Also a result, the current tile 210 has dynamically configured the nexttile 210, and so on, resulting in dynamic self-configuration andself-reconfiguration in each HTF circuit cluster 205.

Numerous advantages of the representative embodiments are readilyapparent. The representative apparatus, system and methods provide for acomputing architecture capable of providing high performance and energyefficient solutions for compute-intensive kernels, such as forcomputation of Fast Fourier Transforms (FFTs) and finite impulseresponse (FIR) filters used in sensing, communication, and analyticapplications, such as synthetic aperture radar, 5G base stations, andgraph analytic applications such as graph clustering using spectraltechniques, machine learning, 5G networking algorithms, and largestencil codes, for example and without limitation.

As used herein, a “processor” 110, 300 may be any type of processor orcontroller, and may be embodied as one or more processor(s) 110, 300configured, designed, programmed or otherwise adapted to perform thefunctionality discussed herein. As the term processor or controller isused herein, a processor 110, 300 may include use of a single integratedcircuit (“IC”), or may include use of a plurality of integrated circuitsor other components connected, arranged or grouped together, such ascontrollers, microprocessors, digital signal processors (“DSPs”), arrayprocessors, graphics or image processors, parallel processors, multiplecore processors, custom ICs, application specific integrated circuits(“ASICs”), field programmable gate arrays (“FPGAs”), adaptive computingICs, associated memory (such as RAM, DRAM and ROM), and other ICs andcomponents, whether analog or digital. As a consequence, as used herein,the term processor or controller should be understood to equivalentlymean and include a single IC, or arrangement of custom ICs, ASICs,processors, microprocessors, controllers, FPGAs, adaptive computing ICs,or some other grouping of integrated circuits which perform thefunctions discussed herein, with associated memory, such asmicroprocessor memory or additional RAM, DRAM, SDRAM, SRAM, MRAM, ROM,FLASH, EPROM or E²PROM. A processor 110, 300, with associated memory,may be adapted or configured (via programming, FPGA interconnection, orhard-wiring) to perform the methodology of the invention, as discussedherein. For example, the methodology may be programmed and stored, in aprocessor 110, 300 with its associated memory (and/or memory 125) andother equivalent components, as a set of program instructions or othercode (or equivalent configuration or other program) for subsequentexecution when the processor 110, 300 is operative (i.e., powered on andfunctioning). Equivalently, when the processor 110, 300 may implementedin whole or part as FPGAs, custom ICs and/or ASICs, the FPGAs, customICs or ASICs also may be designed, configured and/or hard-wired toimplement the methodology of the invention. For example, the processor110, 300 may be implemented as an arrangement of analog and/or digitalcircuits, controllers, microprocessors, DSPs and/or ASICs, collectivelyreferred to as a “processor” or “controller”, which are respectivelyhard-wired, programmed, designed, adapted or configured to implement themethodology of the invention, including possibly in conjunction with amemory 125.

The memory 125, 325, which may include a data repository (or database),may be embodied in any number of forms, including within any computer orother machine-readable data storage medium, memory device or otherstorage or communication device for storage or communication ofinformation, currently known or which becomes available in the future,including, but not limited to, a memory integrated circuit (“IC”), ormemory portion of an integrated circuit (such as the resident memorywithin a processor 130 or processor IC), whether volatile ornon-volatile, whether removable or non-removable, including withoutlimitation RAM, FLASH, DRAM, SDRAM, SRAM, MRAM, FeRAM, ROM, EPROM orE²PROM, or any other form of memory device, such as a magnetic harddrive, an optical drive, a magnetic disk or tape drive, a hard diskdrive, other machine-readable storage or memory media such as a floppydisk, a CDROM, a CD-RW, digital versatile disk (DVD) or other opticalmemory, or any other type of memory, storage medium, or data storageapparatus or circuit, which is known or which becomes known, dependingupon the selected embodiment. The memory 125, 325 may be adapted tostore various look up tables, parameters, coefficients, otherinformation and data, programs or instructions (of the software of thepresent invention), and other types of tables such as database tables.

As indicated above, the processor 110, 300 is hard-wired or programmed,using software and data structures of the invention, for example, toperform the methodology of the present invention. As a consequence, thesystem and related methods of the present invention, including thevarious instructions of the configuration memory 160, may be embodied assoftware which provides such programming or other instructions, such asa set of instructions and/or metadata embodied within a non-transitorycomputer readable medium, discussed above. In addition, metadata mayalso be utilized to define the various data structures of a look uptable or a database. Such software may be in the form of source orobject code, by way of example and without limitation. Source codefurther may be compiled into some form of instructions or object code(including assembly language instructions or configuration information).The software, source code or metadata of the present invention may beembodied as any type of code, such as C, C++, Matlab, SystemC, LISA,XML, Java, Brew, SQL and its variations (e.g., SQL 99 or proprietaryversions of SQL), DB2, Oracle, or any other type of programming languagewhich performs the functionality discussed herein, including varioushardware definition or hardware modeling languages (e.g., Verilog, VHDL,RTL) and resulting database files (e.g., GDSII). As a consequence, a“construct”, “program construct”, “software construct” or “software”, asused equivalently herein, means and refers to any programming language,of any kind, with any syntax or signatures, which provides or can beinterpreted to provide the associated functionality or methodologyspecified (when instantiated or loaded into a processor or computer andexecuted, including the processor 110, 300, for example).

The software, metadata, or other source code of the present inventionand any resulting bit file (object code, database, or look up table) maybe embodied within any tangible, non-transitory storage medium, such asany of the computer or other machine-readable data storage media, ascomputer-readable instructions, data structures, program modules orother data, such as discussed above with respect to the memory 125, 325,e.g., a floppy disk, a CDROM, a CD-RW, a DVD, a magnetic hard drive, anoptical drive, or any other type of data storage apparatus or medium, asmentioned above.

The communication interface(s) 130 are utilized for appropriateconnection to a relevant channel, network or bus; for example, thecommunication interface(s) 130 may provide impedance matching, driversand other functions for a wireline or wireless interface, may providedemodulation and analog to digital conversion for a wireless interface,and may provide a physical interface, respectively, for the processor110, 300 and/or memory 125, with other devices. In general, thecommunication interface(s) 130 are used to receive and transmit data,depending upon the selected embodiment, such as program instructions,parameters, configuration information, control messages, data and otherpertinent information.

The communication interface(s) 130 may be implemented as known or maybecome known in the art, to provide data communication between the HTF200 and/or processor 110, 300 and any type of network or externaldevice, such as wireless, optical, or wireline, and using any applicablestandard (e.g., one of the various PCI, USB, RJ 45, Ethernet (FastEthernet, Gigabit Ethernet, 300ase-TX, 300ase-FX, etc.), IEEE 802.11,Bluetooth, WCDMA, WiFi, GSM, GPRS, EDGE, 3G and the other standards andsystems mentioned above, for example and without limitation), and mayinclude impedance matching capability, voltage translation for a lowvoltage processor to interface with a higher voltage control bus,wireline or wireless transceivers, and various switching mechanisms(e.g., transistors) to turn various lines or connectors on or off inresponse to signaling from processor 130. In addition, the communicationinterface(s) 130 may also be configured and/or adapted to receive and/ortransmit signals externally to the system 100, such as throughhard-wiring or RF or infrared signaling, for example, to receiveinformation in real-time for output on a display, for example. Thecommunication interface(s) 130 may provide connection to any type of busor network structure or medium, using any selected architecture. By wayof example and without limitation, such architectures include IndustryStandard Architecture (ISA) bus, Enhanced ISA (EISA) bus, Micro ChannelArchitecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, SANbus, or any other communication or signaling medium, such as Ethernet,ISDN, T1, satellite, wireless, and so on.

The present disclosure is to be considered as an exemplification of theprinciples of the invention and is not intended to limit the inventionto the specific embodiments illustrated. In this respect, it is to beunderstood that the invention is not limited in its application to thedetails of construction and to the arrangements of components set forthabove and below, illustrated in the drawings, or as described in theexamples. Systems, methods and apparatuses consistent with the presentinvention are capable of other embodiments and of being practiced andcarried out in various ways.

Although the invention has been described with respect to specificembodiments thereof, these embodiments are merely illustrative and notrestrictive of the invention. In the description herein, numerousspecific details are provided, such as examples of electroniccomponents, electronic and structural connections, materials, andstructural variations, to provide a thorough understanding ofembodiments of the present invention. One skilled in the relevant artwill recognize, however, that an embodiment of the invention can bepracticed without one or more of the specific details, or with otherapparatus, systems, assemblies, components, materials, parts, etc. Inother instances, well-known structures, materials, or operations are notspecifically shown or described in detail to avoid obscuring aspects ofembodiments of the present invention. In addition, the various Figuresare not drawn to scale and should not be regarded as limiting.

Reference throughout this specification to “one embodiment”, “anembodiment”, or a specific “embodiment” means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment of the present invention and notnecessarily in all embodiments, and further, are not necessarilyreferring to the same embodiment. Furthermore, the particular features,structures, or characteristics of any specific embodiment of the presentinvention may be combined in any suitable manner and in any suitablecombination with one or more other embodiments, including the use ofselected features without corresponding use of other features. Inaddition, many modifications may be made to adapt a particularapplication, situation or material to the essential scope and spirit ofthe present invention. It is to be understood that other variations andmodifications of the embodiments of the present invention described andillustrated herein are possible in light of the teachings herein and areto be considered part of the spirit and scope of the present invention.

For the recitation of numeric ranges herein, each intervening numberthere between with the same degree of precision is explicitlycontemplated. For example, for the range of 6-9, the numbers 7 and 8 arecontemplated in addition to 6 and 9, and for the range 6.0-7.0, thenumber 6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8, 6.9, and 7.0 areexplicitly contemplated. In addition, every intervening sub-range withinrange is contemplated, in any combination, and is within the scope ofthe disclosure. For example, for the range of 5-10, the sub-ranges 5-6,5-7, 5-8, 5-9, 6-7, 6-8, 6-9, 6-10, 7-8, 7-9, 7-10, 8-9, 8-10, and 9-10are contemplated and within the scope of the disclosed range.

It will also be appreciated that one or more of the elements depicted inthe Figures can also be implemented in a more separate or integratedmanner, or even removed or rendered inoperable in certain cases, as maybe useful in accordance with a particular application. Integrally formedcombinations of components are also within the scope of the invention,particularly for embodiments in which a separation or combination ofdiscrete components is unclear or indiscernible. In addition, use of theterm “coupled” herein, including in its various forms such as “coupling”or “couplable”, means and includes any direct or indirect electrical,structural or magnetic coupling, connection or attachment, or adaptationor capability for such a direct or indirect electrical, structural ormagnetic coupling, connection or attachment, including integrally formedcomponents and components which are coupled via or through anothercomponent.

With respect to signals, we refer herein to parameters that “represent”a given metric or are “representative” of a given metric, where a metricis a measure of a state of at least part of the regulator or its inputsor outputs. A parameter is considered to represent a metric if it isrelated to the metric directly enough that regulating the parameter willsatisfactorily regulate the metric. A parameter may be considered to bean acceptable representation of a metric if it represents a multiple orfraction of the metric.

Furthermore, any signal arrows in the drawings/Figures should beconsidered only exemplary, and not limiting, unless otherwisespecifically noted. Combinations of components of steps will also beconsidered within the scope of the present invention, particularly wherethe ability to separate or combine is unclear or foreseeable. Thedisjunctive term “or”, as used herein and throughout the claims thatfollow, is generally intended to mean “and/or”, having both conjunctiveand disjunctive meanings (and is not confined to an “exclusive or”meaning), unless otherwise indicated. As used in the description hereinand throughout the claims that follow, “a”, “an”, and “the” includeplural references unless the context clearly dictates otherwise. Also asused in the description herein and throughout the claims that follow,the meaning of “in” includes “in” and “on” unless the context clearlydictates otherwise.

The foregoing description of illustrated embodiments of the presentinvention, including what is described in the summary or in theabstract, is not intended to be exhaustive or to limit the invention tothe precise forms disclosed herein. From the foregoing, it will beobserved that numerous variations, modifications and substitutions areintended and may be effected without departing from the spirit and scopeof the novel concept of the invention. It is to be understood that nolimitation with respect to the specific methods and apparatusillustrated herein is intended or should be inferred. It is, of course,intended to cover by the appended claims all such modifications as fallwithin the scope of the claims.

It is claimed:
 1. A configurable circuit, comprising: a configurablecomputation circuit; a plurality of synchronous network inputs coupledto the configurable computation circuit; an asynchronous network outputqueue coupled to an asynchronous packet network; a flow control circuitcoupled to the asynchronous network output queue, the flow controlcircuit configured to generate a stop signal when a predeterminedthreshold has been reached in the asynchronous network output queue; anda configuration memory circuit coupled to the configurable computationcircuit and to the plurality of synchronous network inputs, theconfiguration memory circuit comprising: a first instruction memoryconfigured to store a first plurality of data path configurationinstructions to configure a data path of the configurable computationcircuit; and a second instruction memory configured to store a secondplurality of data path configuration instructions or instruction indicesfor selection of a master synchronous network input of the plurality ofsynchronous network inputs for receipt of a current data pathconfiguration instruction or instruction index from an adjacentconfigurable circuit.
 2. The configurable circuit of claim 1, wherein inresponse to the stop signal, the asynchronous network output queue isconfigured to stop outputting any data packets on the asynchronouspacket network.
 3. The configurable circuit of claim 1, wherein inresponse to the stop signal, the configurable computation circuit isconfigured to stop executing upon completion of a current instruction.4. The configurable circuit of claim 1, wherein the flow control circuitfurther comprises: a plurality of OR gates to receive one or more stopsignals from other configurable circuits of a plurality of theconfigurable circuits arranged in an array; and a stop signalcommunication line to assert the stop signal, the stop signalcommunication line directly coupled to all of the other configurablecircuits of the plurality of configurable circuits of the array.
 5. Theconfigurable circuit of claim 1, wherein the configurable circuitfurther comprises: a plurality of synchronous network outputs coupled tothe configurable computation circuit; wherein a plurality of theconfigurable circuits are arranged in an array, wherein for eachconfigurable circuit of the plurality of the configurable circuits ofthe array, the plurality of synchronous network outputs are directlycoupled to the plurality of synchronous network inputs of adjacentconfigurable circuits of the plurality of configurable circuits of thearray to form a synchronous network.
 6. The configurable circuit ofclaim 1, wherein the second plurality of data path configurationinstructions or instruction indices further comprises data pathconfiguration instructions or instruction indices for selection of thecurrent data path configuration instruction of the first plurality ofdata path configuration instructions from the first instruction memoryfor the configurable computation circuit.
 7. The configurable circuit ofclaim 6, wherein the configurable circuit further comprises: aconfiguration memory multiplexer coupled to the first instruction memoryand to the second instruction memory, the configuration memorymultiplexer having a selection input, wherein when the selection has afirst setting, the current data path configuration instruction isselected from the first plurality of data path configurationinstructions of the first instruction memory using a selected data pathconfiguration instruction or instruction index from the secondinstruction memory, and when the selection input has a second setting,the second setting different from the first setting, the current datapath configuration instruction is selected using the data pathconfiguration instruction or instruction index received on the mastersynchronous network input.
 8. The configurable circuit of claim 1,wherein the configurable circuit further comprises: a plurality ofsynchronous network outputs coupled to the configurable computationcircuit; wherein the second plurality of data path configurationinstructions or instruction indices further comprises data pathconfiguration instructions or instruction indices for selection of anext data path configuration instruction or instruction index for outputto the master synchronous network input of a next configurablecomputation circuit.
 9. The configurable circuit of claim 8, furthercomprising: a conditional logic circuit coupled to the configurablecomputation circuit, the conditional logic circuit configured to provideconditional branching depending upon an output from the configurablecomputation circuit by providing a first next data path configurationinstruction or instruction index for output to the master synchronousnetwork input of the next configurable circuit or providing a different,second next data path configuration instruction or instruction index foroutput to the master synchronous network input of the next configurablecircuit.
 10. The configurable circuit of claim 1, further comprising: acontrol circuit coupled to the configurable computation circuit, thecontrol circuit comprising: a plurality of control registers configuredto store, in a completion table, a plurality of thread identifiers and,for each thread identifier of the plurality of thread identifiers, acorresponding data completion count; and a thread control circuitconfigured to self-schedule and queue an execution thread for executionupon a predetermined cadence when, for the thread identifier for theexecution thread, the corresponding data completion count indicatescompletion of all data dependencies.
 11. The configurable circuit ofclaim 1, further comprising: a control circuit coupled to theconfigurable computation circuit, the control circuit comprising: aplurality of control registers configured to store a completion tablehaving a plurality of thread identifiers, a corresponding datacompletion count for each thread identifier of the plurality of threadidentifiers and, for a thread identifier of the plurality of threadidentifiers for a current execution thread, an identification of a lastiteration or a next thread identifier of the plurality of threadidentifiers identifying a next execution thread for execution followingexecution of the current execution thread to provide ordered threadexecution; and a thread control circuit configured to queue a selectedexecution thread for execution when, for the thread identifier for theselected execution thread, the corresponding data completion countindicates completion of all data dependencies and the thread identifierfor the selected execution thread identifies the selected executionthread as the next execution thread for execution.
 12. The configurablecircuit of claim 1, further comprising: a control circuit coupled to theconfigurable computation circuit, the control circuit comprising: athread control circuit; and a plurality of control registers configuredto store a completion table having a plurality of types of threadidentifiers, with each type of thread identifier indicating a loop levelfor loop and nested loop execution.
 13. The configurable circuit ofclaim 1, further comprising: a thread control circuit comprising: a runqueue configured to store one or more thread identifiers for one or morethreads which are ready to execute; a continuation queue configured tostore one or more thread identifiers for threads having data completioncounts allowing execution or threads without an assigned threadidentifier; and a reenter queue configured to store one or more threadidentifiers for threads having data completion counts allowing executionto provide for continuing execution of the threads in the reenter queueby moving the threads in the reenter queue into the run queue ahead ofany threads in the continuation queue.
 14. The configurable circuit ofclaim 1, further comprising: a control circuit coupled to theconfigurable computation circuit, the control circuit comprising: aplurality of control registers configured to store a thread identifierpool having a predetermined number of a plurality of thread identifiersand a completion table having a loop count of an active number of loopthreads; and a thread control circuit, wherein in response to receipt ofan asynchronous packet network message returning a thread identifier ofthe plurality of thread identifiers to the thread identifier pool, thecontrol circuit is configured to decrement the loop count and, when theloop count reaches zero, to transmit an asynchronous packet networkcompletion message.
 15. The configurable circuit of claim 1, furthercomprising: an interconnection network; a processor coupled to theinterconnection network; a host interface circuit coupled to theinterconnection network; a plurality of the configurable circuitsarranged in an array; and a synchronous network coupled to eachconfigurable circuit of the plurality of configurable circuits of thearray; wherein the asynchronous packet network is coupled to eachconfigurable circuit of the plurality of configurable circuits of thearray.
 16. The configurable circuit of claim 1, further comprising: aninterconnection network comprising a first plurality of crossbarswitches; a processor coupled to the interconnection network; aplurality of configurable circuit clusters coupled to theinterconnection network, each configurable circuit cluster of theplurality of configurable circuit clusters comprising: a plurality ofthe configurable circuits arranged in an array; a synchronous networkcoupled to each configurable circuit of the plurality of configurablecircuits of the array, the synchronous network providing a plurality ofdirect connections and at least one pass through connection betweenadjacent configurable circuits of the plurality of configurable circuitsof the array; a memory interface circuit coupled to the asynchronouspacket network and to the interconnection network; and a dispatchinterface circuit coupled to the asynchronous packet network and to theinterconnection network; wherein the asynchronous packet networkcomprises a second plurality of crossbar switches, each crossbar switchof the second plurality of crossbar switches coupled to at least oneconfigurable circuit of the plurality of configurable circuits of thearray and to another crossbar switch of the second plurality of crossbarswitches.
 17. A system comprising: an asynchronous packet network; asynchronous network; and a plurality of configurable circuits arrangedin an array, each configurable circuit of the plurality of configurablecircuits comprising: a configurable computation circuit; an asynchronousnetwork input queue coupled to the asynchronous packet network; anasynchronous network output queue coupled to the asynchronous packetnetwork; a plurality of synchronous network inputs coupled to theconfigurable computation circuit; a plurality of synchronous networkoutputs coupled to the configurable computation circuit; a flow controlcircuit coupled to the asynchronous network output queue, the flowcontrol circuit configured to generate a stop signal when apredetermined threshold has been reached in the asynchronous networkoutput queue; and a configuration memory circuit comprising: a firstinstruction memory configured to store a first plurality of data pathconfiguration instructions to configure an internal data path of theconfigurable computation circuit; and a second instruction memoryconfigured to store a second plurality of data path configurationinstructions or instruction indices for selection of a current data pathconfiguration instruction of the first plurality of data pathconfiguration instructions from the first instruction memory, forselection of a master synchronous network input of the plurality ofsynchronous network inputs for receipt of the current data pathconfiguration instruction or instruction index from an adjacentconfigurable circuit of the plurality of configurable circuits of thearray, and for selection of a next data path configuration instructionor instruction index for output to the master synchronous network inputof a next configurable circuit of the plurality of configurable circuitsof the array.
 18. The system of claim 17, wherein in response to thestop signal, the asynchronous network output queue of each configurablecircuit of the plurality of configurable circuits of the array isconfigured to stop outputting data packets on the asynchronous packetnetwork, and each configurable computation circuit of each configurablecircuit of the plurality of configurable circuits of the array isconfigured to stop executing upon completion of a current instruction.19. The system of claim 17, wherein each configurable circuit of theplurality of configurable circuits of the array further comprises: acontrol circuit coupled to the configurable computation circuit, thecontrol circuit comprising: a plurality of control registers configuredto store a completion table having a plurality of thread identifiers, acorresponding data completion count for each thread identifier of theplurality of thread identifiers and, for each thread identifier for acurrent thread, of the plurality of thread identifiers, anidentification of a last iteration or another thread identifier of theplurality of thread identifiers identifying a next thread for executionfollowing execution of the current thread; and a thread control circuitconfigured to queue a thread for execution when, for the threadidentifier for the thread, the corresponding data completion countindicates completion of any data dependencies and the thread identifieris identified as the last iteration or the next thread.
 20. A systemcomprising: an interconnection network; a processor coupled to theinterconnection network; a synchronous network; an asynchronous packetnetwork; and a plurality of configurable circuits arranged in an array,of each configurable circuit of the plurality of configurable circuitsof the array comprising: a configurable computation circuit; a pluralityof synchronous network inputs and outputs coupled to the configurablecomputation circuit and to the synchronous network; an asynchronousnetwork input queue and an asynchronous network output queue coupled tothe asynchronous packet network; a flow control circuit coupled to theasynchronous network output queue, the flow control circuit configuredto generate a stop signal when a predetermined threshold has beenreached in the asynchronous network output queue, wherein in response tothe stop signal, the configurable computation circuit is configured tostop executing upon completion of a current instruction and theasynchronous network output queue is configured to stop outputting datapackets on the asynchronous packet network; a thread control circuitconfigured to self-schedule and queue a thread for execution by theconfigurable computation circuit upon a predetermined cadence; and aconfiguration memory circuit coupled to the configurable computationcircuit, to the plurality of synchronous network inputs, and to theplurality of synchronous network outputs, the configuration memorycircuit comprising: a first instruction memory configured to store afirst plurality of data path configuration instructions to configure adata path of the configurable computation circuit; and a secondinstruction memory configured to store a second plurality of data pathconfiguration instructions or instruction indices for selection of acurrent data path configuration instruction of the first plurality ofdata path configuration instructions from the first instruction memory,and for selection of a master synchronous network input of the pluralityof synchronous network inputs for receipt of the current data pathconfiguration instruction or instruction index from an adjacentconfigurable circuit of the plurality of configurable circuits of thearray.